-- Logs begin at Fri 2022-08-26 05:17:42 UTC, end at Fri 2022-08-26 06:37:34 UTC. -- Aug 26 05:53:53.143777 dv-cinder-853910-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 05:53:53.152628 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 05:53:53 2022] *** Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: nodename: dv-cinder-853910-3-iscsi Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: machine: x86_64 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: clock source: unix Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: pcre jit disabled Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: detected number of CPU cores: 4 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: current working directory: / Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: detected binary path: /usr/bin/uwsgi-core Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: your processes number limit is 47680 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: your memory page size is 4096 bytes Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: detected max file descriptor number: 2048 Aug 26 05:53:53.154751 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: lock engine: pthread robust mutexes Aug 26 05:53:53.155675 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: thunder lock: enabled Aug 26 05:53:53.155675 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 05:53:53.155675 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 05:53:53.196026 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: Python main interpreter initialized at 0x55709840d9c0 Aug 26 05:53:53.196026 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: python threads support enabled Aug 26 05:53:53.196026 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: your server socket listen backlog is limited to 100 connections Aug 26 05:53:53.196026 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: your mercy for graceful operations on workers is 90 seconds Aug 26 05:53:53.196438 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 05:53:53.196515 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: *** Operational MODE: preforking *** Aug 26 05:53:53.196753 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: *** uWSGI is running in multiple interpreter mode *** Aug 26 05:53:53.196753 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: spawned uWSGI master process (pid: 81747) Aug 26 05:53:53.196970 dv-cinder-853910-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 05:53:53.198091 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: spawned uWSGI worker 1 (pid: 81761, cores: 1) Aug 26 05:53:53.198091 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: spawned uWSGI worker 2 (pid: 81762, cores: 1) Aug 26 05:53:53.198091 dv-cinder-853910-3-iscsi devstack@keystone.service[81747]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 05:53:57.590492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 05:53:57.590492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: from cryptography.utils import int_from_bytes Aug 26 05:53:57.591580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 05:53:57.591580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: from cryptography.utils import int_from_bytes Aug 26 05:53:57.592330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 05:53:57.592330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: from cryptography.utils import int_from_bytes Aug 26 05:53:57.592789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 05:53:57.592789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: from cryptography.utils import int_from_bytes Aug 26 05:53:57.642082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] command line args: [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ================================================================================ {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] admin_token = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.647844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] config_dir = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.648228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 05:53:57.648300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.648365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 05:53:57.648428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] config_source = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.648527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] command line args: [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 05:53:57.648630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] debug = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.648630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 05:53:57.649187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ================================================================================ {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 05:53:57.649412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] admin_token = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] config_dir = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] config_source = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.649942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] debug = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_config_append = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_dir = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_file = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.650854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_config_append = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_dir = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_file = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.651992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.652989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] publish_errors = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.653850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] publish_errors = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] standard_threads = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.654934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] use_journal = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] use_json = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] use_stderr = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] use_syslog = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.655964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] standard_threads = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.656425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.656702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] use_journal = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.656898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] use_json = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.656959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] use_stderr = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.657016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] use_syslog = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.657122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 05:53:57.657242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.657958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.658989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.659920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.660852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.661818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.662166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.662372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.662565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.662791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.662983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.663170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.663355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.663579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.663775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.663968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.664927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.665978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.666975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.667924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.668956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.669982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.670966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.671912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.672959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.673962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.674947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.675956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.676999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.677930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.678979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.679870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.external = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.680989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.password = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] auth.token = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.681961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.external = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.682908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.password = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] auth.token = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.683885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.684809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.685993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.686917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.687950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.688924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.689903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.690598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.693923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.694959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695536 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.695995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.696952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.697941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.698972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.699888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.700138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.700378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.700553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.701303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.701468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.701588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.701773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.701902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.702988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.703922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.704974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.705918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.706933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.707979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.708905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.709933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.710917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.711935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] role.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] role.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.712974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.713910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.714977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.715935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.716052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.716490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.716672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.716796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.716967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.717983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.718952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.719994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] role.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] role.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.720919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.721930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.722694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.729931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.730948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.connection = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.731993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.732969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.733966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81761) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.734832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.connection = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.735991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.736412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.736412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.736607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.736858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.737104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 05:53:57.737279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81762) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 05:53:57.760750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.760975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.761200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.763121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.763366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.763593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.770034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.772431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.799691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.799902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.802341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.802575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.861511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.861709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.941320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.941678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.944068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:57.944408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.116628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.116798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.117998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.118070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.118070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.118190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.153114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.153337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.153584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.154967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.155082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.155255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.155346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.155536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.155759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.166560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.167706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.179734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.179966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.180194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.180415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.180633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.180848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.180976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.181063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81762) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.181222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.181456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.181707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.181941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.182182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.182405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81761) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 05:53:58.211571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.211770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 05:53:58.212107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.212310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 05:53:58.212586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.212668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.212852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.212937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 05:53:58.213247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.213383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.213467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 05:53:58.213650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.213776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.214055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.214683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.215016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.223633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.223919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.224195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.224482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.225826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.226205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.226515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.226877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.231617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.235070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.235189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.235427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.235715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.236004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.236291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.236555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.236828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.237105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.237498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.237788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.238065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.238145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.238411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.238744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.239071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.239364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.239674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.239982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.245257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.245257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.245257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.245257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.260790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.262276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.262563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.265405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.267004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.267306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.267391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.267681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.267972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.272355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.272696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.273009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.274378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.274693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.279742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.280079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.280368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.280653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.280736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.280934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.281016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.281214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.281487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.281760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.285459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.285762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.286056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.286350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.286693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.287020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.287408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.287684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.299418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.306449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.306449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.306841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.307143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.307228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.307476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.311930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.312352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.315189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.315909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.316248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.318422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.318753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.319050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.319340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.319640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.319925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.320726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.324254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.329092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.329414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.329769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.330090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.330391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.330741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.336656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.336965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.337249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.337518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.344829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.346377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.346693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.347658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.347979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.348272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.348582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.349350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.349640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.349941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.350229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.350511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.356058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.357805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.358174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.361067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.361406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.367713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.367815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.375252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.375602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.375908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.376198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.376490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.376866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.377177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.380078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.383150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.386038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.387515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.389742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.390028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.390315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.394215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.396890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.397452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.398363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.400552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.401544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.401836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.407656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.407656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.407882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.407882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.407882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.407882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.410679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.410975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.411255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.412013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.413525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.416664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.416957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.417247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.417577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.418259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81762) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.418545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.418849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.419134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.419551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.419843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.420123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.420408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.420702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.420984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.421257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.421522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.421809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.422076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81762) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.426602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.426978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.427270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.434637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81761) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 05:53:58.435477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.435789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.436095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.436391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.436706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.437004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.437290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.437605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.437890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.438169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.438438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.438758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.439073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81761) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 05:53:58.456958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55709840d9c0 pid: 81762 (default app) Aug 26 05:53:58.460618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111d2b-4712-44a0-8924-bf112e286270 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:53:58.460909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111d2b-4712-44a0-8924-bf112e286270 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:53:58.461184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2111d2b-4712-44a0-8924-bf112e286270 None None] PATH_INFO: `/v3` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:53:58.465339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Fri Aug 26 05:53:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:53:58.476638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55709840d9c0 pid: 81761 (default app) Aug 26 05:54:06.198908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:06.199543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:06.199543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:06.200954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 2/2] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:54:06.209423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:06.209894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:06.210049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:06.286882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: INFO dbcounter [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] Registered counter for database keystone Aug 26 05:54:06.314103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG oslo_db.sqlalchemy.engines [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81761) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 05:54:06.320707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writer thread running {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 05:54:06.522290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG passlib.handlers.bcrypt [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81761) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 05:54:06.522669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG passlib.handlers.bcrypt [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81761) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 05:54:06.580322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:06.582006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:06.737178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1/3] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 26 05:54:07.239565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:07.311163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: INFO dbcounter [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] Registered counter for database keystone Aug 26 05:54:07.331651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG oslo_db.sqlalchemy.engines [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81762) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 05:54:07.338026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writer thread running {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 05:54:07.473852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15cc66-bcc7-4820-ae6d-34b5012e263d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:07.478334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:07.479002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:07.479288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:07.479563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:07.488272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:07.541529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 05:54:07.541895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ec018-1f8d-4681-8d46-92488ffc80fe admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:07.552677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 3/4] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 26 05:54:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 05:54:07.558824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:07.573950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728e51f-ccf9-4b6a-a68e-c466cb51983b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:07.579062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:07.580161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:07.580443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:07.580715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] PATH_INFO: `/v3/projects/6316e285f13041a3ae4690b463c7dfa7` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:07.582191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] RBAC: Authorizing `identity:get_project(project_id=6316e285f13041a3ae4690b463c7dfa7)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:07.636036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 05:54:07.636386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c9a01-bb53-4898-8819-05dacf45246d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:07.641672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 2/5] 10.222.0.40 () {56 vars in 1207 bytes} [Fri Aug 26 05:54:07 2022] GET /identity/v3/projects/6316e285f13041a3ae4690b463c7dfa7 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:09.572585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80c7-93b4-4a59-862b-9f3103a22a1f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:09.573136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80c7-93b4-4a59-862b-9f3103a22a1f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:09.573136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c80c7-93b4-4a59-862b-9f3103a22a1f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:09.577219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 4/6] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:09.582764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:09.583143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:09.583323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:09.603263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:09.604807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:09.613509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 3/7] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 05:54:09.891219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c80c7-93b4-4a59-862b-9f3103a22a1f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:09.903597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c80c7-93b4-4a59-862b-9f3103a22a1f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:09.908024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:09.908633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:09.908919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:09.909193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:09.917404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:09.919457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a243e-4d04-4ce2-89d0-ddea899c2a40 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:09.928095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 5/8] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Aug 26 05:54:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 05:54:09.934149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:09.944579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc1075-2d38-4f5c-aa7b-3eb19d502c80 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:09.948924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:09.949552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:09.949837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:09.951257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:09.952580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:09.954430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19019c07-11e9-4c47-87f7-e329d260a322 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:09.955979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 4/9] 10.222.0.40 () {56 vars in 1198 bytes} [Fri Aug 26 05:54:09 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:11.970748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1b55-e127-43e9-aa74-a48f0ec66fa9 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:11.971294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1b55-e127-43e9-aa74-a48f0ec66fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:11.971294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1b55-e127-43e9-aa74-a48f0ec66fa9 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:11.972656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 6/10] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:11.978436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:11.978975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:11.979068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:11.997754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:11.999288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:12.000472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1131b5-2404-4ef3-9b0b-ca3ea6a396c2 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.000762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1131b5-2404-4ef3-9b0b-ca3ea6a396c2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.001040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1131b5-2404-4ef3-9b0b-ca3ea6a396c2 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.002479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 7/11] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:12.007818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.008103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.008375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.053673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG passlib.handlers.bcrypt [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81762) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 05:54:12.054090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG passlib.handlers.bcrypt [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81762) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 05:54:12.084825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 5/12] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:12.101337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:12.103830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:12.219420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 8/13] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:12.371563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.398786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:12.491415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120c6ef-a940-4694-9c48-d764f8113bdb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:12.495409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.528072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:12.528731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.529019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.529303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.541126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:12.550999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:12.552027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:12.552830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 6/14] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:12.558029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.605740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edc4dc-dbde-4fdb-a101-4364bf807f0f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:12.608772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:12.652062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:12.652631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:12.652969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.653280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.653554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.673164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd4fd0-153c-44c4-97d8-0542a2948394 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:12.709036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:12.709665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.709954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.710234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.710951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:12.711260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:12.712677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:12.725502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 7/15] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:12.729402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:12.731013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:12.731206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.838609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 9/16] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:54:12.840105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2055-3033-4bb3-b6ac-97554e1127f8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:12.844949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.848635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:12.892505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:12.893158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:12.893446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:12.893728 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:12.895011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:12.896488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:12.898153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 8/17] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:12.904624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:12.921355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:12.969052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680faba1-f7f0-4641-9614-f9da36c3f7e6 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:13.008130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:13.009020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:13.009308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:13.009586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:13.014188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d7322-7be5-42d2-a808-3bbfbf5bdb68 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:13.030820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:13.032414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:13.051076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:13.051721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:13.052002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:13.052282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:13.053445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:54:13.053557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:13.054505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:54:13.055488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:54:13.055488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:54:13.055488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:13.055488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:54:13.055782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:13.056667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:13.060713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:13.063031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:13.122735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:13.162455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:13.174068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05303513-0d75-4534-bbfc-cdce6dd3fab8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:13.182173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134958-a657-46a1-b5d2-837c064281c4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:13.209166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:13.209783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:13.210070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:13.210351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:13.211036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:13.211343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:13.212756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6529cd3-cbd5-442f-a06d-7b7444058696 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:13.219543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:13.220164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:13.220452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:13.220731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:13.221470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.domain.id=default, user.id=d4b059129fcd4293b7ae6f76c1ca5b3e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:13.221783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:13.223290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a0b81-ce92-44d5-ba7b-2787336707af None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:13.230478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 10/20] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 26 05:54:13 2022] GET /identity/v3/role_assignments?user.id=d4b059129fcd4293b7ae6f76c1ca5b3e&scope.domain.id=default&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:13.235190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 11/21] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:15.337470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9994f2-fa63-4cf9-8c3b-37d48b55a01e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:15.372573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9994f2-fa63-4cf9-8c3b-37d48b55a01e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:15.372573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9994f2-fa63-4cf9-8c3b-37d48b55a01e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:15.372573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 11/22] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:15.373177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96a1c-421e-47ea-b112-9fc8a71ef002 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:15.373177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96a1c-421e-47ea-b112-9fc8a71ef002 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:15.373177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96a1c-421e-47ea-b112-9fc8a71ef002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:15.375709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2fa96a1c-421e-47ea-b112-9fc8a71ef002 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:15.376929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2fa96a1c-421e-47ea-b112-9fc8a71ef002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:15.480623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 12/23] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:15.784685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f737c4-26f6-495b-8292-c278cf1ae17f None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:15.785059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f737c4-26f6-495b-8292-c278cf1ae17f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:15.785243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f737c4-26f6-495b-8292-c278cf1ae17f None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:15.786675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 12/24] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:15.792387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5688-01f9-48c5-8292-40d62cbb5074 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:15.794642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5688-01f9-48c5-8292-40d62cbb5074 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:15.794642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b5688-01f9-48c5-8292-40d62cbb5074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:15.827680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-335b5688-01f9-48c5-8292-40d62cbb5074 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:15.828919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-335b5688-01f9-48c5-8292-40d62cbb5074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:15.899162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:15.936309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 13/25] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:16.022407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f737c4-26f6-495b-8292-c278cf1ae17f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:16.037294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cc30a-dc46-47f4-a778-c748b8282e23 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.037605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cc30a-dc46-47f4-a778-c748b8282e23 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.037772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84cc30a-dc46-47f4-a778-c748b8282e23 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.039210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 14/26] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:16.055587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983c7ff-1463-424a-a464-7b612fddd715 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.055897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983c7ff-1463-424a-a464-7b612fddd715 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.056150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983c7ff-1463-424a-a464-7b612fddd715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.089648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4983c7ff-1463-424a-a464-7b612fddd715 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:16.093592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4983c7ff-1463-424a-a464-7b612fddd715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:16.227922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 15/27] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:16.231785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2a0ad-ec5f-41cb-b54f-47e8e8f241fb None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.232744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2a0ad-ec5f-41cb-b54f-47e8e8f241fb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.233810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2a0ad-ec5f-41cb-b54f-47e8e8f241fb None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.235993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f737c4-26f6-495b-8292-c278cf1ae17f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:16.238131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 16/28] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:16.241627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.242174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.246579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.273023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:16.275026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:16.323463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:16.324057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.324348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.324632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.355761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:16.360400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 17/29] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:16.362897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:16.364519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:16.365284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:16.366775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 13/30] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:16.378281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:16.397599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:16.402985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:16.538376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac859376-08d9-4338-8271-b826dcf40d81 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:16.547010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:16.591810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:16.592463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.592757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.593044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.593800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:16.595284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:16.690454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917b3a0-2a99-441e-a4c1-e302672cd91e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:16.729879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:16.730501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.730817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.731976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.732721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:16.733029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:16.734430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:16.749482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 18/31] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:16.750339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:16.865862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:16.874877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834ddd-8047-4076-b6fb-b41f16e77e3a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:16.875280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 14/32] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 05:54:16 2022] POST /identity/v3/roles => generated 219 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:16.876625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:16.937059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:16.937776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:16.938055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:16.938328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:16.961375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:16.962945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:16.966341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:54:16.994252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:16.995209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:54:16.995997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:54:16.995997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:54:16.995997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:16.995997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 19/33] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:16.996213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.015844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119fac1-2b4f-4e63-a6cd-bf2186135d3f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.056146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.057191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.057691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.058253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.059318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.061260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.144057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 15/34] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 05:54:16 2022] POST /identity/v3/roles => generated 225 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:17.144880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.154350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e67c-f89a-461e-90d3-d49e50c5572a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.225758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.226416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.226742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.227072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.228725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.230463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.237270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 20/35] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:16 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:17.243756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.289472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:17.368135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b419d-c293-4961-9116-4908ed97b2b9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.411000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:17.411886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f110c-8005-45b3-9d2c-db55693850c8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.429657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.430635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.431105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.431553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.432662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:17.433150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.435914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c117-db9c-447d-b292-ab58ef32f6c6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.464644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 16/36] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:17.471239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f185fac-b61c-41ba-8de3-b3e26fd481a3 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.476836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f185fac-b61c-41ba-8de3-b3e26fd481a3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.476836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f185fac-b61c-41ba-8de3-b3e26fd481a3 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.477201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.477306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 17/37] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:17.477375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.477375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.477492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f185fac-b61c-41ba-8de3-b3e26fd481a3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.477564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.477564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.478477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1d70-53b0-4804-8ee6-19048fa48539 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.480370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 21/38] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:17.481856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f351-a0f5-429d-b594-152b5394795d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.482150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f351-a0f5-429d-b594-152b5394795d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.482505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f351-a0f5-429d-b594-152b5394795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.507230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c044f351-a0f5-429d-b594-152b5394795d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:17.508525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c044f351-a0f5-429d-b594-152b5394795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:17.586231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 22/39] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:17.587135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044f351-a0f5-429d-b594-152b5394795d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.599456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f185fac-b61c-41ba-8de3-b3e26fd481a3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.607945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:17.645110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.645723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.646018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.646305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.647010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.648930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35d036-fdb9-403d-b3d6-569ea45228fb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.711427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044f351-a0f5-429d-b594-152b5394795d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:17.749013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:17.749677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.749959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.750243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] PATH_INFO: `/v3/domains/default/users/d4b059129fcd4293b7ae6f76c1ca5b3e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.752221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d4b059129fcd4293b7ae6f76c1ca5b3e, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:17.753678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:17.772169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 18/40] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:54:17 2022] POST /identity/v3/projects => generated 301 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:17.773434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd45c1-7fed-4664-abda-c0a27a2bdd71 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.773725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd45c1-7fed-4664-abda-c0a27a2bdd71 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.774009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd45c1-7fed-4664-abda-c0a27a2bdd71 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.775554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 19/41] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:17.782774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:17.783069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:17.783398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:17.804489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:17.806123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:17.849409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:17.897764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 20/42] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:17.898973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:17.949576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 23/43] 10.222.0.40 () {58 vars in 1361 bytes} [Fri Aug 26 05:54:17 2022] PUT /identity/v3/domains/default/users/d4b059129fcd4293b7ae6f76c1ca5b3e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:18.039724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15508551-3caa-4c93-af67-fcdb8e20fc96 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:18.084545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:18.085180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:18.085467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:18.085746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:18.086427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:18.087895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:18.181014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 21/44] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 05:54:17 2022] POST /identity/v3/roles => generated 223 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:18.182670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:18.279736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:18.307681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249db171-93e7-48f7-af5d-b5fa4becaa4b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:18.348990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:18.350039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:18.350662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:18.351479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:18.352746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:18.354447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cad37-f7e4-490b-bb2a-956b0345efd1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:18.360028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 24/45] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:18.365659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:18.503764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:18.592610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4666a7-1a9e-4a06-b82f-9eaebc1b4116 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:18.627225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:18.627885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:18.628191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:18.628475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:18.629146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:18.630651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3ca2d-964e-4372-8bba-7fcd904da95f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:18.749094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 22/46] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:54:18 2022] POST /identity/v3/projects => generated 312 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:54:19.915462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ad08-7a42-4564-b65f-e86b1274c053 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:19.916000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ad08-7a42-4564-b65f-e86b1274c053 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:19.916000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6ad08-7a42-4564-b65f-e86b1274c053 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:19.917380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 25/47] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:19.923821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:19.924284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:19.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:19.942753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:19.943967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:20.015189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 23/48] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:20.296385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ad08-7a42-4564-b65f-e86b1274c053 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:20.406498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6ad08-7a42-4564-b65f-e86b1274c053 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:20.440113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:20.440749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.441031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.441304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.453465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:20.459358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:20.460389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:20.461135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 26/49] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:20.466942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:20.477620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:20.579568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd9148-6544-4490-8f3d-6aa0c6474f11 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:20.616918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:20.617535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.617820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.618098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.618807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:20.619115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:20.620502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deaaac-69f8-4edb-a025-603c4f0f98da None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:20.629933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 24/50] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:20.637705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:20.644614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:20.709148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520358-2e6c-4efe-b738-0ecfe7cc17b8 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.709432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520358-2e6c-4efe-b738-0ecfe7cc17b8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.709710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76520358-2e6c-4efe-b738-0ecfe7cc17b8 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.711240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 25/51] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:54:20.717044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.717331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.717609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.737551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:20.738919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:20.748222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca44487-102e-4abd-bd9d-91a7eb27afd5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:20.785282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:20.785939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.786235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.786516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.787864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:20.789367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:20.791035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 27/52] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:20 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:20.796675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:20.808294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:20.813120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 26/53] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:20.875234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:20.909332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78be84f-d341-4843-b53d-d5f4fbe9f907 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:20.944928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:20.945556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:20.945847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:20.946128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:20.947344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:20.948812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:20.951079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 28/54] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:20.957023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:21.059354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102147b-66e8-4547-ab18-4584b06d0600 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:21.094771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:21.095392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:21.095675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:21.095955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:21.096702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.domain.id=default, user.id=d4b059129fcd4293b7ae6f76c1ca5b3e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:21.097017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:21.098490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:21.098864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:21.104922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 27/55] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Aug 26 05:54:20 2022] GET /identity/v3/role_assignments?user.id=d4b059129fcd4293b7ae6f76c1ca5b3e&scope.domain.id=default&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:21.315875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240572f6-7636-4198-9e9a-b64bec275aea None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:21.352723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:21.353360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:21.353641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:21.353910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:21.355153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:21.356634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42173cb3-f278-4fa0-b316-5cba7cc324d0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:21.358209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 29/56] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:21.363996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:21.428397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:21.467591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b79f03-70ac-4855-a551-5dfefbd0d45c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:21.502926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:21.503647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:21.504137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:21.504644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:21.505920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:21.507466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea7f20-9255-47ab-afc0-a207f4e07ce8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:21.605816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 28/57] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 26 05:54:21 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:23.535957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93479fa-2ab0-4a55-bbae-6461784afb3c None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:23.536511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93479fa-2ab0-4a55-bbae-6461784afb3c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:23.536511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93479fa-2ab0-4a55-bbae-6461784afb3c None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:23.537863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 30/58] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:23.543235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:23.543941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:23.543941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:23.562449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:23.563687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:23.632287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 29/59] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:23.914339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93479fa-2ab0-4a55-bbae-6461784afb3c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:23.925561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:24.019240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93479fa-2ab0-4a55-bbae-6461784afb3c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:24.053852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:24.054485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:24.054804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:24.055092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:24.056225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:24.057677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b8239-3f1b-48a7-ad12-4a48d8c8e0c0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:24.059336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 31/60] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:24.065114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:24.112247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:24.168209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c4ca4-6702-4188-b431-201c7e61219b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:24.204867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:24.205484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:24.205775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:24.206060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:24.206751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:24.208223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065b0e4-2be9-4488-8cbe-375d3b6f5ee4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:24.316621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 30/61] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:24 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:27.412227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164033c3-b1d4-452f-91a8-786ad7945df3 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.412775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164033c3-b1d4-452f-91a8-786ad7945df3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.412775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164033c3-b1d4-452f-91a8-786ad7945df3 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.414305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 32/62] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:27.432559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067df87-cd91-4ea9-bcd8-4faa289575bc None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.433398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067df87-cd91-4ea9-bcd8-4faa289575bc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.433861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067df87-cd91-4ea9-bcd8-4faa289575bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.460126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f067df87-cd91-4ea9-bcd8-4faa289575bc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:27.461364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f067df87-cd91-4ea9-bcd8-4faa289575bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:27.538655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a44cd-2f0b-4684-87f8-bb0ec35e1a47 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.538655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a44cd-2f0b-4684-87f8-bb0ec35e1a47 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.538655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a44cd-2f0b-4684-87f8-bb0ec35e1a47 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.539526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 33/63] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:27.547560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34bee-d4d8-4287-b7ae-bf468c8dcecb None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.548134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34bee-d4d8-4287-b7ae-bf468c8dcecb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.548577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f34bee-d4d8-4287-b7ae-bf468c8dcecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.571448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-88f34bee-d4d8-4287-b7ae-bf468c8dcecb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:27.573378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88f34bee-d4d8-4287-b7ae-bf468c8dcecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:27.586982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 31/64] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:27.665069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 34/65] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:27.851412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25d1f-298d-486c-867e-84c5bf434f3e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.852094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25d1f-298d-486c-867e-84c5bf434f3e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.852582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25d1f-298d-486c-867e-84c5bf434f3e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.854170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 32/66] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:27.859944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e382415-633d-42c9-a46a-3ab35bad4136 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.860527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e382415-633d-42c9-a46a-3ab35bad4136 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.861007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e382415-633d-42c9-a46a-3ab35bad4136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.879735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:27.885652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9e382415-633d-42c9-a46a-3ab35bad4136 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:27.887532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9e382415-633d-42c9-a46a-3ab35bad4136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:27.906960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25d1f-298d-486c-867e-84c5bf434f3e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:27.945668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:27.980311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 35/67] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:27.988399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb20b5-35d0-4e32-8036-5a9c86457ca1 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:27.988885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb20b5-35d0-4e32-8036-5a9c86457ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:27.989314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb20b5-35d0-4e32-8036-5a9c86457ca1 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:27.991095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 36/68] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:28.003804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.004303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.004747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.028333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25d1f-298d-486c-867e-84c5bf434f3e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.055862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:28.057175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:28.073359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.074607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.075200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.075699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.091147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:28.098456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:28.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:28.100181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:28.101258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 33/69] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:54:28.101258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.172146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 37/70] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 05:54:28.173362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.222204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.233142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0694c0-a814-423a-a6ea-5f601cc7b88b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.248182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.275293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.277388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.278280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.279316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.284800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41e0a-1f47-421d-80a8-8b2693de5293 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.292783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-bb23a2b8-d971-4c05-a7bb-6ee55c4bcabc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:28.298661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:28.299509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:28.300274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:28.302184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 34/71] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:28.303661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37887-0e70-412e-b648-b76594519390 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.304172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37887-0e70-412e-b648-b76594519390 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.304647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37887-0e70-412e-b648-b76594519390 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.306470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 35/72] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:28.311796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.312451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.312948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.332402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.333079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.333371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.333652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.334316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:28.334832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:28.335392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.336253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:28.337186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:28.353248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 38/73] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:28.355244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.429199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 36/74] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:28.430182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.482408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc170d-038b-4c46-bab6-db3c47a1f0c1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.510484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.519663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.520294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.520583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.522154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.522154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:28.522154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.523432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:28.532908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 39/75] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:28.533570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.539377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.562950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8f42e-91d9-46d8-bb98-115b9a6934df None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.602790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.603446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.603738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.604066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.605303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.607456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:28.609288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 37/76] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:28.609916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.649968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e26d8-18d6-493e-b1a2-4e19f2194227 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.689193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.690124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.690475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.690792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.702294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:28.707099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:28.707965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:28.708713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 40/77] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:28.709335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.729208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17732e7f-1cc2-4c18-b6c8-e8d70f17dee5 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.762286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.765575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.766228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.766517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.766836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.768154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.769625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:28.771241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 38/78] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:28.779722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.792043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.814110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763c5c-ec6e-4dc8-b3e6-a3cd6adbb0a1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.852398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.853044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.853343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.853626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.870580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.872156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:28.873883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 41/79] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:28.874673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:28.882924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a29621-5080-461a-87b8-da8106e47bdf None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.924146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:28.924772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:28.925060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:28.925340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:28.980194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9eb5d6-431d-4b58-b376-3ec2b2b1f1f7 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:28.989667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:28.998261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:28.999803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.016058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.016657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.016939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.017213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.019852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.028983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:29.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:29.041533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 42/80] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:29.042298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.112497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 39/81] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:29.117917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.149541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282c871-ac7d-413c-8707-7c378d20cffc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.184300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.184915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.185207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.185483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.186156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:29.186460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.187887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.196086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 43/82] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:29.196911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.228483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fb66a-8420-404a-8d80-a5d13d78f1d7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.233869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.245924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.265689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.266523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.266769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.267060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.267720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.269241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.304999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49979ec2-dd3d-488d-90db-bb4a462a21de None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.343394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.344010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.344293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.344565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.405759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 40/83] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:29 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:29.406633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.417625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.419511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.456971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.534192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86af6-429f-4e42-85e9-a74e3ac95e63 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.550254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 44/84] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:29.551784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.577829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.578514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.579344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.579344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.579802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:29.580112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.581532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.592182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 41/85] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:29.601938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.644613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.679962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2df833-bee9-460f-8495-64ced69ff4c3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.726356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:29.733376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.734027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.734324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.734587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.737415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.737712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.738285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727b7ef-caf8-4dbe-bfff-f3fb1bd3a930 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.739596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 45/86] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:29.740172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.792284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.793927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.794521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.795069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.796684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.798402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.800480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 42/87] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:29.801629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.897077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7431-1120-465f-b7cb-84fb4acf3b7c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.944057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:29.944993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:29.945284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:29.945563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:29.946917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:29.948907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:29.952542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 46/88] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:29.953535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:29.961416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48de6de-ea26-45f1-9ed7-030f6ae79e6e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:29.964280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.013735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.015020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.015318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.015605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.016843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.019444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.022145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 43/89] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:30.023449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.045184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.096000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590657-c262-4576-8396-3437d8d09f4c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:30.144664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.145659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.145944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.146220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.147012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=d4b059129fcd4293b7ae6f76c1ca5b3e) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:30.147332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.148815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.166816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 47/90] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:29 2022] GET /identity/v3/role_assignments?user.id=d4b059129fcd4293b7ae6f76c1ca5b3e&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:30.168082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.186555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf28a2-53b6-4699-a392-626b711494ba None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:30.239132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.239774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.240228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.240637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.297302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.330488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.333218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.401242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf3cb-c40b-455f-9e90-2e033924cd33 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:30.450909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.467994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 44/91] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:30.468750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.468855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.471141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.471141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.471141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.471141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:30.471401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.472944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.480266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 48/92] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:30.481122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.575197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.618977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5d2a-45d6-4399-a0f4-3cb165bdce60 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:30.669896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.671158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.671447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.671732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.712778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438f535-6bc9-4e8b-aad4-83b8c228d72e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:30.759462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.760976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.817713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:30.818859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:30.819401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:30.819933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:30.823430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:30.825995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:30.828124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 49/93] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:30.844969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.895091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:30.926495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 45/94] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:30.936308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:30.967789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:31.006180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0c71d-028d-4fa6-89e9-1de9c9b4925a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:31.044171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:31.044807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:31.045100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:31.045380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:31.046137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:31.046453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:31.047965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:31.054334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 50/95] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:31.119410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91eea1-4796-47c9-a00d-9bad53a76752 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:31.188228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:31.188903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:31.189195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:31.189481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] PATH_INFO: `/v3/projects/c894807ad31543dfae23c3452d1cc180` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:31.190766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] RBAC: Authorizing `identity:get_project(project_id=c894807ad31543dfae23c3452d1cc180)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:31.192317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f51c489-2278-4a19-909d-c2329bd1c5ac None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:31.194063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 46/96] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:30 2022] GET /identity/v3/projects/c894807ad31543dfae23c3452d1cc180 => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:31.201167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:31.322225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ce14-2ac0-46bf-994e-0aab28240d28 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:31.355040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:31.360389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:31.361018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:31.361306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:31.361591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:31.362372 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=c894807ad31543dfae23c3452d1cc180, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:31.362720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:31.364211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59df8a2-92aa-4030-a780-384d16502366 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:31.376518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 51/97] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:31 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=c894807ad31543dfae23c3452d1cc180&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:32.220270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7428-0607-4dfe-a685-715254ff648c None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:32.221125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7428-0607-4dfe-a685-715254ff648c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:32.221275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7428-0607-4dfe-a685-715254ff648c None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:32.223426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 47/98] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:32.234946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:32.235785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:32.235785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:32.262701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:32.263984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:32.312897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40528bd0-93cf-477d-8949-00b4ae8d0e5f None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:32.313303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40528bd0-93cf-477d-8949-00b4ae8d0e5f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:32.313471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40528bd0-93cf-477d-8949-00b4ae8d0e5f None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:32.315048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 48/99] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:32.340819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:32.341084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:32.341486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:32.369211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 52/100] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:32.379908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:32.381164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:32.465616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:32.545186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 49/101] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:32.964979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.164300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf397f-c016-4fc4-986c-3d55550b39b7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:33.203456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:33.226788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:33.228273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.229106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.229878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.260816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:33.272598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:33.274069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:33.275398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:33.284397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 53/102] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:33.291780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.426643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b0047-2585-45c9-a294-96c3d047c75d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:33.463449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:33.464082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.465420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.465420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.465969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:33.466281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:33.467745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:33.468385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.477050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 50/103] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:33.485122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.531146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:33.672365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a282a-3cf1-4143-a0b1-b05de1f5702a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:33.674539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24de75-635d-4074-b752-d5a30f525051 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:33.722436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:33.723447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.723990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.724481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.726644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:33.729297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3b140-1ad2-4e9d-88b6-02172b6c0f4b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:33.733220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 54/104] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:33.734519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2167c1-c243-4733-b008-6a70c30bcb8a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.734835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2167c1-c243-4733-b008-6a70c30bcb8a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.735232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2167c1-c243-4733-b008-6a70c30bcb8a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.737226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 55/105] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:33.742812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:33.742983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2167c1-c243-4733-b008-6a70c30bcb8a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.743495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.743794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.744083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] PATH_INFO: `/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.776402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2fcd7bb09245f0998042b5c48b4dbb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:33.778044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b596fe-4e9c-4003-80b0-050b78337023 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:33.780030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 51/106] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:33 2022] GET /identity/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:33.781019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.781320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.781592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.831743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:33.835166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:33.856906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:33.928290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2167c1-c243-4733-b008-6a70c30bcb8a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:33.930237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 52/107] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:33.931586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:33.956499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:33.978943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:33.978943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:33.979488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:33.979782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:33.980870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:33.983374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e2335-eec1-4aaf-a611-0065b707d292 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:34.055136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca202bc5-ef16-4913-b441-7fcff224b073 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:34.078468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 56/108] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:54:33 2022] POST /identity/v3/projects => generated 302 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:34.080105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881118-9519-419d-9228-7f1d34a2f10c None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.081164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881118-9519-419d-9228-7f1d34a2f10c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.081679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881118-9519-419d-9228-7f1d34a2f10c None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.086538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 57/109] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:34.092931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.093505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.093929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.130974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:34.132729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:34.134057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:34.134888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.135311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.138203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.266023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:34.271257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:34.286583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 58/110] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:34.287429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:34.316909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:34.342812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:34.431148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 53/111] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:34.437960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:34.454342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b40f9-52d4-4a37-b3d8-27ca57be083b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:34.498566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:34.499222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.499665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.499957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.520922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-6a439b71-51ee-414d-bb37-aa70e231dbe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:34.526386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:34.527369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:34.528116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 59/112] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:34.529333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796522-b9e2-4e78-9312-f661dd777736 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.529333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796522-b9e2-4e78-9312-f661dd777736 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.529539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796522-b9e2-4e78-9312-f661dd777736 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.531285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 60/113] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:34.537664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796522-b9e2-4e78-9312-f661dd777736 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:34.585064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523e2af-7e2a-476c-87e4-9094f063dff1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:34.611634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:34.630591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:34.631370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.631831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.632195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.634140 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:34.636412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19c837-35ff-4e1d-a22a-48bb347906e6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:34.639287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 54/114] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:34 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:34.646244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.646881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.647182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.670275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:34.672895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:34.718018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:34.729641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796522-b9e2-4e78-9312-f661dd777736 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:34.774715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:34.775354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:34.775928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:34.776204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:34.776936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:34.777336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:34.779368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:34.784774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 55/115] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:34.786077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:34.790657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 61/116] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:34.796839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.005524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3178-e2da-4b5f-9b13-ce71aed54dde None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.008948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167afcfc-4c3d-498b-9b3e-2469d635f9c3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.039787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.063896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.064227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.064783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.065073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.065349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.065717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.066180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.066663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.067916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=5d2fcd7bb09245f0998042b5c48b4dbb) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:35.068847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.070539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.078704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 56/117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:34 2022] GET /identity/v3/role_assignments?user.id=5d2fcd7bb09245f0998042b5c48b4dbb&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=035acef499d346fdaeadda8d604f717e => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.080542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:35.080803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5ea61813-f805-4070-9d56-38f059fa8bd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:35.087015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:35.087972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:35.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 62/118] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:35.095673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3d0b-b677-47a0-a55e-41f22e53c466 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.096080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3d0b-b677-47a0-a55e-41f22e53c466 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.096443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea3d0b-b677-47a0-a55e-41f22e53c466 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.102084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 63/119] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:35.102084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea3d0b-b677-47a0-a55e-41f22e53c466 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.118770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.234088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bb83b-39c9-424f-b847-a005aeeca3b0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.249154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea3d0b-b677-47a0-a55e-41f22e53c466 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.273889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.274540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.274940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.275189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.276619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.280690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b58fd-107c-4ecb-85cf-a7cf64662e1a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.282420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 57/120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.288426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.288778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.289055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.291236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.292430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.292941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.293415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.294296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:35.294867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.296506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.306395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 64/121] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.307567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.312683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:35.314092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:35.381109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.415212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 58/122] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:35.418829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.427436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.472648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16162-8858-4cc9-93fc-6f9563872612 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.545742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.546323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.546630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.546918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.574292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:35.584301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:35.585210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:35.585964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 65/123] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:35.586589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.615472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d308df-718b-471f-ac4a-1660034cc776 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.663409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.664120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.664416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.664703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.665992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.667502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.670752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 59/124] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.671578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.774435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.799337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e0ced-8201-4172-85b6-1c8d8788d845 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.836344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc54e3-ff48-4a78-9659-c47069656d26 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:35.844258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.844915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.845204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.845488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.847021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.848528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.850343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 66/125] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.854725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:35.858363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:35.890430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:35.891699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:35.892515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:35.893275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:35.894201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:35.894759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:35.897536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:35.914709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 60/126] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:35.915837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.024586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d501-c250-4ee2-adbe-648a7e545cfe None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.050473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244d75b-e0b6-4bb2-a4fb-6f46bee990cb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.070666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:36.075795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.078580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.079892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.083155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/d4b059129fcd4293b7ae6f76c1ca5b3e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.091501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user_id=d4b059129fcd4293b7ae6f76c1ca5b3e, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.093552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.096199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.097030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.097601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.097912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.112321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:36.117944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:36.121435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:36.122204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:36.123225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 61/127] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:36.123781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.213560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 67/128] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:35 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/d4b059129fcd4293b7ae6f76c1ca5b3e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:36.219528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.375327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519207e-f45d-4cdb-aa52-55148c0ed1b8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.421228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:36.439790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.440515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.440784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.441054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.444630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.446184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.456559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 62/129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:36 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:36.458131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.487089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeb815-3412-4ee3-9ba3-dd781ebb7039 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.512618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:36.582721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb6681-31c5-4b72-aebc-5e66851b014c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.609477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.610303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.610602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.610908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.612098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.613710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.616683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 68/130] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:36 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:36.621819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.627767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.628323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.628609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.629190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.630388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:36.631059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.634509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6495bdc-33d5-4334-94a9-b4cd45241de7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.645449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 63/131] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:36.651687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dde4f-974b-4764-b28b-bf1ba102f27d None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.652575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dde4f-974b-4764-b28b-bf1ba102f27d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.652872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dde4f-974b-4764-b28b-bf1ba102f27d None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.654466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 64/132] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:36.659608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dde4f-974b-4764-b28b-bf1ba102f27d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.803256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b716d-7899-4a2d-ac14-feb8662f5f35 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.805259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:36.815124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dde4f-974b-4764-b28b-bf1ba102f27d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:36.840577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:36.847039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.848013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.848477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.848941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/9a35490ab66448dc9fc3db978a411b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.851426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user_id=9a35490ab66448dc9fc3db978a411b2f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.853139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947270f-6e26-486e-aca6-e952fb1777a8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.868513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:36.870882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.870882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.870882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.871290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:36.874181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:36.875025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 65/133] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:36 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:36.881468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:36.957207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 69/134] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:36 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/9a35490ab66448dc9fc3db978a411b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:36.962042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:36.962042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:36.962042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:36.990368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:36.992523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:37.073700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b26a7-52e5-4a16-8f3f-a4c605067bcb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:37.148207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:37.148791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.149080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.149360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.151820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:37.153891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:37.157592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 70/135] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:37.158345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.163070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 66/136] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:36 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:37.169936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.238301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:37.319998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe191c-67c2-4821-a92e-1fbabf6cf25d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:37.358732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:37.359515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.359679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.359967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] PATH_INFO: `/v3/projects/c894807ad31543dfae23c3452d1cc180` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.362124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] RBAC: Authorizing `identity:get_project(project_id=c894807ad31543dfae23c3452d1cc180)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:37.363746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a498b-48c9-42f1-9583-cddc33f54af5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:37.365798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 67/137] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:37 2022] GET /identity/v3/projects/c894807ad31543dfae23c3452d1cc180 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:37.367123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4edc-1a79-4dd0-89fa-d0c46d600265 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.367413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4edc-1a79-4dd0-89fa-d0c46d600265 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.367723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4edc-1a79-4dd0-89fa-d0c46d600265 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.368949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 68/138] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:37.377603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.377893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.378173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.403976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d03ab9-ace8-49ed-8556-d86cf8e9ac88 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:37.405907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:37.407223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:37.426421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:37.442286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:37.443016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.443306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.443591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/9a35490ab66448dc9fc3db978a411b2f/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.453629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user_id=9a35490ab66448dc9fc3db978a411b2f, role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:37.457040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:37.506362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 69/139] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:37.507316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.540794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 71/140] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:37 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/9a35490ab66448dc9fc3db978a411b2f/roles/c17cdb7e94834eee81063576196e3fd9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:37.558726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.596987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:37.735225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab956-0366-4e8c-a5e6-a4ec9d6af6fe None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:37.785175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394a414-1ece-4832-b8a9-9ca87c0ebd86 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:37.801056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:37.802101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.802564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.803603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] PATH_INFO: `/v3/projects/c894807ad31543dfae23c3452d1cc180/users/9a35490ab66448dc9fc3db978a411b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.807048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] RBAC: Authorizing `identity:create_grant(project_id=c894807ad31543dfae23c3452d1cc180, user_id=9a35490ab66448dc9fc3db978a411b2f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:37.809000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:37.860038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:37.861393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:37.862100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:37.862893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:37.864954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:37.866694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:37.868548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 72/141] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:37.875114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.888073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 70/142] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:37 2022] PUT /identity/v3/projects/c894807ad31543dfae23c3452d1cc180/users/9a35490ab66448dc9fc3db978a411b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:37.909372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:37.965052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:38.046716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dd9d6-325d-4f6c-b876-317d9e9961b1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:38.123468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e33be0-a6eb-48be-aabf-a99614e04a56 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:38.126987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:38.146383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:38.147044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:38.147329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:38.147608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:38.148293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:38.149816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:38.211818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:38.212493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:38.212787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:38.213065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:38.230922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:38.236432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:38.237708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:38.238490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 71/143] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:38.243835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:38.280705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 73/144] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:37 2022] POST /identity/v3/users => generated 283 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:54:38.397269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008bc4f7-ea47-46fc-96ff-b4471200b744 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:38.435937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:38.436580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:38.436863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:38.437193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:38.437891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:38.438201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:38.439688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:38.454656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 72/145] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:38.462964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:38.594231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7131b0-f2a7-454c-a916-96f9851ca776 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:38.596934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:38.633331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:38.633983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:38.634325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:38.634607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] PATH_INFO: `/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:38.635871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] RBAC: Authorizing `identity:get_user(user_id=5d2fcd7bb09245f0998042b5c48b4dbb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:38.637823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:38.639914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 74/146] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:38 2022] GET /identity/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:38.645616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:38.669304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:38.770599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ee6cf-7206-495e-9546-8409a9ac258b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:38.813677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:38.814326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:38.814612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:38.814970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:38.819446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:38.820975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaed605-6e4b-4fce-bdb9-a319bea44a20 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:38.823090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 73/147] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:38 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:38.831876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:38.955263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c052d5-a536-49cf-800a-1601b08f88ce None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:39.003302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:39.004039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.004322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.004602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/5d2fcd7bb09245f0998042b5c48b4dbb/roles/035acef499d346fdaeadda8d604f717e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.006606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user_id=5d2fcd7bb09245f0998042b5c48b4dbb, role_id=035acef499d346fdaeadda8d604f717e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:39.008106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:39.032465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41b777-d363-4f09-bfc7-3850aee998e9 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.032780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41b777-d363-4f09-bfc7-3850aee998e9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.033037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41b777-d363-4f09-bfc7-3850aee998e9 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.034458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 74/148] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:54:39.040063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0703-d9a8-4320-923f-27822fa632b4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.040355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0703-d9a8-4320-923f-27822fa632b4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.040640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0703-d9a8-4320-923f-27822fa632b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.064121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-82dd0703-d9a8-4320-923f-27822fa632b4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:39.065343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-82dd0703-d9a8-4320-923f-27822fa632b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:39.116342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 75/149] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:38 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/users/5d2fcd7bb09245f0998042b5c48b4dbb/roles/035acef499d346fdaeadda8d604f717e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:39.173008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 75/150] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:39.639793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:39.683704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:39.746499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131044f8-e6c4-4c6d-9e61-1eae7f133424 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.747147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131044f8-e6c4-4c6d-9e61-1eae7f133424 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.747147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131044f8-e6c4-4c6d-9e61-1eae7f133424 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.748463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 76/151] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:39.755879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.756188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.756455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.795573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:39.796791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:39.816529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:39.821765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca9159-d5e9-4933-8fef-eac1ad1a30ed None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:39.882930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:39.883634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:39.883921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:39.884194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:39.914864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:39.923466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:39.924391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:39.925132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:39.927216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 76/152] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:39.935069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:39.952453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 77/153] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:40.075630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:40.088724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de851b-b10d-4bc1-a987-34e3c8a7d356 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:40.159371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:40.160582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:40.161084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:40.161543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:40.162426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:40.163110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:40.164772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:40.182287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 77/154] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:40.188400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:40.316114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478efd-311b-495e-81b1-fb42bcf8cc7a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:40.364788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:40.365822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:40.366280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:40.366775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] PATH_INFO: `/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:40.368137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] RBAC: Authorizing `identity:get_user(user_id=d4b059129fcd4293b7ae6f76c1ca5b3e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:40.369801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:40.372190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 78/155] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:40 2022] GET /identity/v3/users/d4b059129fcd4293b7ae6f76c1ca5b3e => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:40.378113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:40.509063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28706fc-0b85-419c-be6a-16721f63d9dd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:40.546295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:40.547241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:40.547733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:40.548231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:40.581995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:40.638692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:40.640805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:40.759750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 78/156] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:40.771584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:40.789301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:40.793998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:40.907019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:40.954510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37e699-ccde-4299-a7ae-a67ff10495e2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.020775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d673c87-d424-4cc7-a253-274d55e52c1f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.039100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.039892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.040182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.040459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.044627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:41.046930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:41.049108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 79/157] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:40 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:41.055999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:41.101980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.102851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.103141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.103417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.118657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:41.129259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:41.130168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:41.131401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 79/158] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:41.139421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:41.284592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:41.297297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a265e66-68ff-497b-9560-1e7e32e5c978 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.319043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:41.354263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.354950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.355230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.357374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.358291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:41.358761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:41.360307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:41.361560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de928a-f85b-4bc2-b4bf-e8e816a90d7b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.369907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 80/159] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:41.377142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:41.471883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.472923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.473420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.473870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.475254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=d4b059129fcd4293b7ae6f76c1ca5b3e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:41.475804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:41.477474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fbeb5-4b49-4217-b6cc-a35c2d605f3b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:41.492947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 80/160] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:41 2022] GET /identity/v3/role_assignments?user.id=d4b059129fcd4293b7ae6f76c1ca5b3e&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:41.494911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c2f4-2d80-4524-a004-23c8140fa6ee None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.496131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c2f4-2d80-4524-a004-23c8140fa6ee None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.496827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383c2f4-2d80-4524-a004-23c8140fa6ee None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.503637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 81/161] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:41.509542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20350-59b1-48be-a6e8-e2eec0ad018d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.510029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20350-59b1-48be-a6e8-e2eec0ad018d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.510463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20350-59b1-48be-a6e8-e2eec0ad018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.510872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a411-6b09-4b10-894c-bcc915787b30 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.554193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-20a20350-59b1-48be-a6e8-e2eec0ad018d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:41.556538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-20a20350-59b1-48be-a6e8-e2eec0ad018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:41.565623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.566764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.567356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.567831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.569296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:41.571033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01ac0e-9b6d-4d34-a91c-982ab50a8bb4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:41.573068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 81/162] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:41 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:41.575319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d09b9-5c1b-4a91-839b-6b1ecebbbda6 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.575829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d09b9-5c1b-4a91-839b-6b1ecebbbda6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.576319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d09b9-5c1b-4a91-839b-6b1ecebbbda6 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.577908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 82/163] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:41.583793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d09b9-5c1b-4a91-839b-6b1ecebbbda6 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:41.675044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:41.757387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d09b9-5c1b-4a91-839b-6b1ecebbbda6 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:41.757878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 82/164] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:41.759253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.759547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.759826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.791644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:41.793559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:41.800633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:41.807511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:41.807511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:41.807511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:41.916890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:41.923015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:41.983079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:41.996961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 83/165] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:42.054832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 83/166] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:42.058848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.062435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:42.195952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e72010-5895-4292-bf88-a5ca88c23c1f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:42.237676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:42.238597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:42.239111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:42.239592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:42.240941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:42.242442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:42.244194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 84/167] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:42.253306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.345725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.389677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50f31-97f9-4c8c-99a0-b1499d0c369b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:42.430590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:42.431602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:42.432107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:42.432588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:42.434549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:42.436748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:42.438439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:42.445459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 84/168] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:42.476958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:42.519766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.544874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:42.552615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756360f-065a-4986-8132-7ba02e97b140 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:42.690085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4db4b-dad8-4637-8e8d-96a944d138d3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:42.691949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:42.692769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:42.692876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:42.693183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:42.743417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:42.746770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:42.747730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:42.748215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:42.748604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:42.749444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:42.750223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 85/169] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:42.751577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:42.757418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.774229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:42.781246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:42.782145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:42.783035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:42.790644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 85/170] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:54:42.792059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:42.891535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:42.969180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcb9d3-484c-4b8c-9178-4a404bf64f3d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:42.981649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd210ae-ee15-4b17-8b64-e4915caf9f79 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.021649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:43.036372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.037010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.037292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.037674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.038375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:43.038707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.040187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.053547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.054264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.054645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.054924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.055357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 86/171] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:43.055724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:43.056023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.057581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.063152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:43.077327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 86/172] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:43.083323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:43.216107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d77673-bcac-496c-b52b-b347940e6bc7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.275198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.276034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.276421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.276784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.280745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df8640-eca8-44d4-8075-3f0ef453d173 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.283155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.284958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f6ac7-f519-45da-8cea-afb92d9e1dfe None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.286862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 87/173] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:43 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:43.290894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff863-4898-4762-87df-a068c63e6774 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.291208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff863-4898-4762-87df-a068c63e6774 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.291500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff863-4898-4762-87df-a068c63e6774 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.293284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 88/174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:43.294490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827474a-a61c-4664-908a-aa3a2c60c88f None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.294837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827474a-a61c-4664-908a-aa3a2c60c88f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.295116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827474a-a61c-4664-908a-aa3a2c60c88f None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.296465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 89/175] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:43.301560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827474a-a61c-4664-908a-aa3a2c60c88f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:43.315201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:43.320532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] PATH_INFO: `/v3/users/9a35490ab66448dc9fc3db978a411b2f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] RBAC: Authorizing `identity:get_user(user_id=9a35490ab66448dc9fc3db978a411b2f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592e190-b77e-4553-8ea3-618b7a8aaf99 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.327991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 87/176] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:43 2022] GET /identity/v3/users/9a35490ab66448dc9fc3db978a411b2f => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:43.328398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a17-b107-4314-b10b-f89b4037f670 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.328398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a17-b107-4314-b10b-f89b4037f670 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.328652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a17-b107-4314-b10b-f89b4037f670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.349543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-fe496a17-b107-4314-b10b-f89b4037f670 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:43.350911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fe496a17-b107-4314-b10b-f89b4037f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:43.382360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:43.427585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 88/177] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:43.431179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.431179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.431179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.434053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827474a-a61c-4664-908a-aa3a2c60c88f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.456582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:43.459278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:43.485683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.486414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.486911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.487262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.580591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 89/178] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:43.581349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:43.615399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.617082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cfc2d-07cf-4f0c-8bcf-97adff70b928 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.659107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:43.712158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:43.764538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1844e-d7ae-41e3-8351-6b759bcf2f54 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.774073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 90/179] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:43.775861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b534-521c-4446-a696-fc76ae4070f7 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.776154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b534-521c-4446-a696-fc76ae4070f7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.776435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0b534-521c-4446-a696-fc76ae4070f7 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.779030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 91/180] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:43.781628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0b534-521c-4446-a696-fc76ae4070f7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:43.840775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.841307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.841588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.841858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.937493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0b534-521c-4446-a696-fc76ae4070f7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:43.943338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.944844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e794d5d-caca-4bf8-9d59-1d39badd0f6a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.976328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:43.976971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.977244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.977518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:43.978656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:43.980837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752626f-064c-42f8-b2a4-1a14a2c1345e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:43.982507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 92/181] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:43 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:43.988253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:43.988571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:43.988854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.008395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:44.008591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.009616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:44.033903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.057104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 90/182] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:44.058991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250d458-ecab-423e-9493-90fe774e38ca None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.059279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250d458-ecab-423e-9493-90fe774e38ca None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.059550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250d458-ecab-423e-9493-90fe774e38ca None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.060850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 91/183] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:44.061577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250d458-ecab-423e-9493-90fe774e38ca None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.086318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 93/184] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:44.087302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.199238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250d458-ecab-423e-9493-90fe774e38ca None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.202847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf3215-63a7-4796-977a-9c7f0c6327ff None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.242541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.243220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.243501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.243774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.246196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.248331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.248626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.248910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.256929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:44.261848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:44.262915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 92/185] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:44.263915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.273497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:44.279044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:44.279953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:44.280735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 94/186] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:44.281739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.296760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.306015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.396051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7b0b4-7a51-4815-a2ac-649247258c09 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.404549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5a0c-9928-4702-84cd-2dedcabb8dee None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.432404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.433071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.433735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.433735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.434453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:44.434788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.436257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b71e6-9e76-47f5-b19d-588128b7f57d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.442322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 93/187] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.443560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.443687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.443877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.444153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.444248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.444495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.444773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] PATH_INFO: `/v3/projects/c894807ad31543dfae23c3452d1cc180` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.445955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] RBAC: Authorizing `identity:get_project(project_id=c894807ad31543dfae23c3452d1cc180)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.447485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.450332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 95/188] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/projects/c894807ad31543dfae23c3452d1cc180 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.451991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.470001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:44.471251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:44.558442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 94/189] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:44.559151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.580656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.581536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.596445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790eb51-6d32-4909-8e75-4add7330eb45 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.653492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.654716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.655207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.657694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.657694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:44.657694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.659213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.671622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 96/190] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.672456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.704279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0835e62-f412-4802-a363-9ebda043b2ff None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.746154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.746794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.747073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.747475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.748251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:44.748551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.749957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.759637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 95/191] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.764874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.801579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44861c-bacb-4c5f-bc0c-d6645ba10550 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.851582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.852330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.852615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.852890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.854010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.855838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.856808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.857592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 97/192] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.862830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.881928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17b8ef-8f4f-43e4-8f4d-c2c468b2634b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:44.884913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:44.920886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:44.921509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:44.921805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:44.922092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:44.924737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=c894807ad31543dfae23c3452d1cc180, user.id=9a35490ab66448dc9fc3db978a411b2f) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:44.924737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:44.924737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:44.930884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 96/193] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/role_assignments?user.id=9a35490ab66448dc9fc3db978a411b2f&scope.project.id=c894807ad31543dfae23c3452d1cc180&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:44.931713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:44.991669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9b40b-6d3f-4337-8772-470891b51108 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.037743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.038408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.038726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.039005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.056566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.058880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.060827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 98/194] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:45.061513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.068386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c6012-4dea-4535-afa4-cd29ba191f1a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.111787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.112434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.112716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.113018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] PATH_INFO: `/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.114191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2fcd7bb09245f0998042b5c48b4dbb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.115688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.117291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 97/195] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:44 2022] GET /identity/v3/users/5d2fcd7bb09245f0998042b5c48b4dbb => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:45.118033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.129678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.176417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.208428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a60e46-9a2a-4272-9bc2-7e9d3d61f597 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.247368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.247922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.248204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.248475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.251864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b3e-e077-477d-ab1a-c27efb1d9951 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.259270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:45.265440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:45.266340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 99/196] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:45.267429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.290921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.291531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.291809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.292080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.292760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.294173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.372714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190726-43b6-45e3-87fe-a689cdc1f39f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.408085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.408710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.408999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.409279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.413097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.427363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 98/197] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:45 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:45.427363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.500377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.503143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.520180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.572374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436197a-c8e6-4b0c-a747-66118d413cd5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.624541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.625186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.625471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.625738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.651531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 100/198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:45.653216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.707135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.720650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.722126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.805348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de196b0-4955-43ad-900a-04945b2bcd8c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:45.806488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:45.863914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:45.865024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:45.865506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:45.865925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:45.866647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:45.867221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:45.869448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:45.872499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 99/199] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:45.873143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:45.881318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 101/200] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:45.886679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.032052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012724e9-af85-4a3c-9bae-30a6f2296265 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.043752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:46.049589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a2347-cea3-4b7d-b0f8-e4ee269b150d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.076190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.076585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.076934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.078203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.079792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.086632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 100/201] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.086632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.110223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.110911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.111484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.111770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.113260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.115464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.122128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 102/202] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:45 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.122128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.129652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:46.219497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e04b1-f0e3-45fb-9045-11cfaa3d4cb0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.252147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593d4d-f419-4e44-b402-eb39c9590512 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.256015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.256669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.256955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.257233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.258434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.259936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.262788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.263683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.299178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.300149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.300658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.301129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.302227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=c57f12edf1bf4f1099ed713e3e3ad12e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:46.302762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.304421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.311024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 103/204] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/role_assignments?user.id=c57f12edf1bf4f1099ed713e3e3ad12e&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.312437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.354435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:46.375293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:46.379329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b603f8-a014-418e-a05d-79583daaee8d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.421731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.422359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.422662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.422944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.423756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69, user.id=5d2fcd7bb09245f0998042b5c48b4dbb) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:46.424071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.425570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.438680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 102/205] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/role_assignments?user.id=5d2fcd7bb09245f0998042b5c48b4dbb&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=035acef499d346fdaeadda8d604f717e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.454250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba131b98-e338-4599-90a8-cc009f46a01d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.519975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.520631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.520918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.521191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.611412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.612859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.656750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:46.735658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 104/206] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:46.743779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:46.864430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfeb6-5717-46d9-9cb4-3a70e6c53798 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:46.915204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:46.915869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:46.916152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:46.916423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:46.918378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:46.920519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce02e6-6d78-486b-878e-391cf2adf481 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:46.924144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 103/207] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:46.929686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:47.055161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7496cca-2d28-4b7a-82c3-a6fa0789e4c0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:47.086637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:47.093533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:47.094153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:47.094433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:47.094732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:47.095474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=c57f12edf1bf4f1099ed713e3e3ad12e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:47.095785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:47.097398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf9f3f-78e6-4380-b9f6-34aa38ff4789 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:47.103398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 105/208] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:46 2022] GET /identity/v3/role_assignments?user.id=c57f12edf1bf4f1099ed713e3e3ad12e&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:47.780535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302de6-e0ed-4d87-9183-45d31ddd1770 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:47.780920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302de6-e0ed-4d87-9183-45d31ddd1770 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:47.781098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302de6-e0ed-4d87-9183-45d31ddd1770 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:47.782439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 104/209] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:47.787802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:47.788096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:47.788380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:47.810585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:47.811825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:47.831891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24f9c-b899-404e-9960-cebdb13ed6fd None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:47.832206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24f9c-b899-404e-9960-cebdb13ed6fd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:47.832445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24f9c-b899-404e-9960-cebdb13ed6fd None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:47.834107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 105/210] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:47.840104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:47.840505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:47.840848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:47.860415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:47.861792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:47.889782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 106/211] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:47.896109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:47.954034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 106/212] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:48.175198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.266152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.296814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4fa9c-e952-4ebb-8f77-9eee1f313b8b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.331127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:48.346221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.347164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.347458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.356379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.361507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:48.368378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:48.369283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:48.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 107/213] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:48.376119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.397211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34e6a3-4236-4cdc-b76b-1c369f8891d3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.463975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.465008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.465495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.465990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.467923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:48.470070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:48.472638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 107/214] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:48.478793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.503149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266631c8-361f-46c8-974c-22046c78e987 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.527351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:48.543725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.545052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.545052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.545398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.546302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:48.546751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:48.548312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:48.563005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 108/215] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:48.564933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.626809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1e80-7bad-4a3a-872a-da885f3c95da None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.633209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:48.668541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.669169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.669454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.669728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.670545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:48.672035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b77f03-ebd2-4ca4-b0b6-5a422f60a455 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:48.694769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580abc-1094-40f9-90c3-9e2b21e95241 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.731179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.731745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.732029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.732307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] PATH_INFO: `/v3/users/8383814a592140cfbe9be293a27e4311` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.747116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] RBAC: Authorizing `identity:get_user(user_id=8383814a592140cfbe9be293a27e4311)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:48.748617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd04efbe-d993-413f-899b-6795ee1b3814 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:48.750338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 109/216] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:48 2022] GET /identity/v3/users/8383814a592140cfbe9be293a27e4311 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:48.751552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de3da-e503-4fc2-b3d7-0967d1204ee5 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.751841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de3da-e503-4fc2-b3d7-0967d1204ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.752116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de3da-e503-4fc2-b3d7-0967d1204ee5 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.753324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 110/217] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:48.756955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3de3da-e503-4fc2-b3d7-0967d1204ee5 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:48.794250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 108/218] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:48 2022] POST /identity/v3/users => generated 297 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:48.795656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bffb3-001d-4435-a524-a928a585b281 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.795950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bffb3-001d-4435-a524-a928a585b281 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.796257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1bffb3-001d-4435-a524-a928a585b281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:48.819457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1c1bffb3-001d-4435-a524-a928a585b281 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:48.820779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1c1bffb3-001d-4435-a524-a928a585b281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:48.944210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3de3da-e503-4fc2-b3d7-0967d1204ee5 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:48.944924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 109/219] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:48.957423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:48.990490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:48.991170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:48.991461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:48.991797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.075320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:49.076908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:49.079697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14ea9-97aa-4551-a196-1f54566862a9 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.079989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14ea9-97aa-4551-a196-1f54566862a9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.080271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14ea9-97aa-4551-a196-1f54566862a9 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.081641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 110/220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:54:49.087741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.089854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.090329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.116472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:49.117735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:49.132191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.226746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 111/221] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:54:49.230124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 111/222] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:49.230653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.234912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.275248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.361348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6292b0-0571-4e71-a224-73c08f1a08bf None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.412324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:49.413262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.413553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.414155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.415952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:49.418115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:49.421125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 112/223] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:49.423297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f01b-39d4-4566-8171-4b9790948fad None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.465794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:49.466897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.467385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.467865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.480214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:49.485781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:49.486715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:49.487499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:49.490225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 112/224] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:49.497322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.507677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.515524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.560967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0411edf-af4d-4e88-ab78-cdcb104671b8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.612384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:49.613407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.614974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.615633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.616831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=8383814a592140cfbe9be293a27e4311) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:49.617355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:49.619067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:49.635019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 113/225] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/role_assignments?user.id=8383814a592140cfbe9be293a27e4311&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:49.635019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.687791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a98ae-d089-4bb3-b661-605f412180bd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.755063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:49.756201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.756684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.757773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.758866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:49.759382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:49.762296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:49.773556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 113/226] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:49.780455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:49.873378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84631-79a9-4c8d-a45f-22a75deb6eef None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.885348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.919002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:49.919662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:49.919953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:49.920236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:49.923241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:49.936350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:49.941183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf666c4-7945-422f-b5f8-0e0964b57299 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:49.941318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:49.942333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:49.943350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 114/227] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:49.951385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.007435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.008742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.009209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.009652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.011919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.013603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.015623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 114/228] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:50.022208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.135102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcf01f-02b5-4489-879d-ba89d484ed3c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.191042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.192176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.192636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.193110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.193957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:50.194432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.196150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.209908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 115/229] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:50.216652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.239103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b269da8-4246-47c1-9700-c54659566fb4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.321774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:50.328270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.328935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.329223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.329538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.331390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.332896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.335193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 115/230] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:50 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:50.342156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.350922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:50.365501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ff4e-b310-452f-833a-0ecc2b175977 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.407396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.407949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.408237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.408520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.409712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.411471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.413422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 116/231] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:50 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:50.419698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.532076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ee40-b634-4d0c-b340-3a52845de280 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.563310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:50.563810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef26ccf-5d51-47d9-8098-f7f9550c9608 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.580584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.581779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.581779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.582034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.583056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.587881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.589818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 117/232] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:50 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:50.595815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:50.663401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.664112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.664399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.664682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/c57f12edf1bf4f1099ed713e3e3ad12e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.667005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, user_id=c57f12edf1bf4f1099ed713e3e3ad12e, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.668494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66290d-ea37-49e4-b32c-1f08c108dc39 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.711684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32894789-893c-4888-9294-42aff7e86ea2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:50.751008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 116/233] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:50 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/c57f12edf1bf4f1099ed713e3e3ad12e/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:54:50.769608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:50.770723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:50.771251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:50.771715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/c57f12edf1bf4f1099ed713e3e3ad12e/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:50.774470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, user_id=c57f12edf1bf4f1099ed713e3e3ad12e, role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:50.776752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8275e-b0f9-4f57-920f-5d10b52405d8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:50.847443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 118/234] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:50 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/c57f12edf1bf4f1099ed713e3e3ad12e/roles/c17cdb7e94834eee81063576196e3fd9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:51.553400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ce08-2608-44ae-a3aa-9febec451870 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:51.553926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ce08-2608-44ae-a3aa-9febec451870 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:51.553926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24ce08-2608-44ae-a3aa-9febec451870 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:51.555425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 117/235] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:51.563345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13888904-7f80-47f2-b1f7-859861fa201d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:51.563925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13888904-7f80-47f2-b1f7-859861fa201d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:51.564465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13888904-7f80-47f2-b1f7-859861fa201d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:51.592767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-13888904-7f80-47f2-b1f7-859861fa201d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:51.593979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-13888904-7f80-47f2-b1f7-859861fa201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:51.605627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac97a8-8922-408f-8612-de70965ef07a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:51.605919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac97a8-8922-408f-8612-de70965ef07a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:51.606200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac97a8-8922-408f-8612-de70965ef07a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:51.607936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 118/236] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:51.612860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56806c-36d1-4a36-b25c-e2af3a2575c3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:51.613156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56806c-36d1-4a36-b25c-e2af3a2575c3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:51.613438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56806c-36d1-4a36-b25c-e2af3a2575c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:51.660329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ae56806c-36d1-4a36-b25c-e2af3a2575c3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:51.661551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ae56806c-36d1-4a36-b25c-e2af3a2575c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:51.678124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:51.697603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 119/237] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:51.765841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:51.790196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 119/238] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:52.010271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedf4b-6ccc-47a5-80e5-e03720b295ac None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.011616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedf4b-6ccc-47a5-80e5-e03720b295ac None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.011923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dedf4b-6ccc-47a5-80e5-e03720b295ac None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.013591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 120/239] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:52.019222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.019579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.019860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.039136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:52.040340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:52.121932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dedf4b-6ccc-47a5-80e5-e03720b295ac None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.136898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 120/240] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:52.141077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.294592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dedf4b-6ccc-47a5-80e5-e03720b295ac None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:52.322446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6dcc66-89b7-4632-b34f-280c28a26399 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:52.336304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:52.336947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.337751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.338042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.351214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:52.357907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:52.358880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:52.359705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 121/241] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:54:52.367935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.384083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:52.412598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:52.414920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.415208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.415486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.418769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:52.425072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:52.428042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 121/242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:52.433654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.466238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:52.535467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d5761a-457c-4b59-830c-3e8874a571b9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:52.571830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a0d38-0ecc-42d2-b671-4faf1e809a6c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:52.588059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:52.590561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.591325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.591746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.593013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:52.593464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:52.595046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:52.607849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 122/243] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:52.609231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.621106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:52.621741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.622025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.622297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.622998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:52.625306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:52.721897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 122/244] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:52 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:52.723229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.750663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968eef2-2509-4e30-9a47-6b497b12f9f9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:52.757151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:52.790657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:52.791344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:52.791702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:52.792602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:52.817466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:52.831055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:52.831978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:52.832755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:52.835853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 123/245] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:52.842907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:52.860997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:52.962603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053e4c6-b31d-4622-9559-a01dbc30cd72 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.003876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418f0d-df8d-4405-becc-1ec2c9cff75f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.008104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.009041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.009527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.010005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] PATH_INFO: `/v3/users/157ff0631d874dbfaebc7ee30bba3242` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.028139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] RBAC: Authorizing `identity:get_user(user_id=157ff0631d874dbfaebc7ee30bba3242)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.030736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.032763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 123/246] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/users/157ff0631d874dbfaebc7ee30bba3242 => generated 297 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:53.039678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.054168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.055250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.056081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.056942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.057929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:53.058514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.060789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.073686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 124/247] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:53.079464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.114925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.166080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0677a2-1891-47af-bc88-67fbf81d5965 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.170036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.206759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714fb96-ff94-4f29-bef0-385f9cb6c84c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.209382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.210732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.210732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.210909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.249598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.250564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.251199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.251800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] PATH_INFO: `/v3/users/8383814a592140cfbe9be293a27e4311` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.253330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=8383814a592140cfbe9be293a27e4311)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.255168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986a00d0-5258-4a02-9c8c-f14a3366e0f6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.258456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 125/248] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/users/8383814a592140cfbe9be293a27e4311 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:53.262847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddedd1-bb34-4386-aba5-b03d4922e806 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.262847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddedd1-bb34-4386-aba5-b03d4922e806 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.262847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddedd1-bb34-4386-aba5-b03d4922e806 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.263200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 126/249] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:53.266447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddedd1-bb34-4386-aba5-b03d4922e806 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.306389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.309288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd249cf-daae-4281-9299-cd4246657b1a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.407949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.415671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddedd1-bb34-4386-aba5-b03d4922e806 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.444679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 124/250] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:53.446078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.446370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.446666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.453962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.457758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.458414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.458740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.459023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.460965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.463219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d77fd-604c-4a7a-b6d2-d97a2f129a86 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.465495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 127/251] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:53.466761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b933e-fa6b-46ab-bc0c-96ee96fa44cd None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.467158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b933e-fa6b-46ab-bc0c-96ee96fa44cd None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.467565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b933e-fa6b-46ab-bc0c-96ee96fa44cd None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.468890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 128/252] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:53.469618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b933e-fa6b-46ab-bc0c-96ee96fa44cd None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.472085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:53.473594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:53.563998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 125/253] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:53.565062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.626007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b933e-fa6b-46ab-bc0c-96ee96fa44cd None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.687959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47665ad-f664-4265-929a-393d1b64b7e8 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:53.714355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.715517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.715803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.716079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.717497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.719684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd74e2b-ee86-437f-8bea-23601b7e5a92 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.722484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 129/254] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:53.729848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.730169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.731331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.741423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:53.742157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:53.742484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:53.742795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/8383814a592140cfbe9be293a27e4311/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:53.746402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, user_id=8383814a592140cfbe9be293a27e4311, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:53.748008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:53.754970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.763495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:53.764791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:53.794917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:53.839392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 130/255] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:53.840320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:53.881721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 126/256] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:53 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/8383814a592140cfbe9be293a27e4311/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:53.946470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.144497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72843a9-cb59-4b12-8ba4-9cdfce541034 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.163134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1237784-7d13-46c4-9a4f-98be9208e095 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.185613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.186252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.186537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.186849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.187606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=157ff0631d874dbfaebc7ee30bba3242) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:54.187924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:54.189497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:54.196876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 131/257] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/role_assignments?user.id=157ff0631d874dbfaebc7ee30bba3242&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=035acef499d346fdaeadda8d604f717e => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:54.196876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.211587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.212236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.212513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.212783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.236179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:54.241012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:54.241914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:54.242701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 127/258] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:54.253706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.275016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:54.330854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:54.383597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc5182-c5f8-4b76-a487-92fd20e14630 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.384649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1534253-84b2-4ca8-9761-ad06c77098ac None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.424497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.425606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.426044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.426156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767b92-3c75-414f-a030-480734034109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.426653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.427059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767b92-3c75-414f-a030-480734034109 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.427409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767b92-3c75-414f-a030-480734034109 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.427752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21767b92-3c75-414f-a030-480734034109 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.428504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21767b92-3c75-414f-a030-480734034109 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:54.428874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21767b92-3c75-414f-a030-480734034109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:54.430389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21767b92-3c75-414f-a030-480734034109 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:54.440515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:54.441115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 128/259] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:54.446161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:54.447191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:54:54.447963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:54:54.449099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767b92-3c75-414f-a030-480734034109 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.451169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 132/260] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:54.458132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.561415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:54.579841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c70438-31df-4707-8868-79d884e04980 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.621285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.622369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.622893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.623361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.624256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:54.624757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:54.626467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:54.637307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 133/261] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:54.644016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.744113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21767b92-3c75-414f-a030-480734034109 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.767064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170a5cd-5dc6-45b8-93f4-608a1b676821 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:54.779689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:54.805735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.806414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.807306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:54.808820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:54.809377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.809858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.811017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:54.811495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] PATH_INFO: `/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:54.813346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:54.815425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] RBAC: Authorizing `identity:get_user(user_id=c57f12edf1bf4f1099ed713e3e3ad12e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:54.817149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:54.819130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:54.821667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 129/262] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:54.822100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 134/263] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/users/c57f12edf1bf4f1099ed713e3e3ad12e => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:54.826052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.835504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:54.886994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:55.082712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba6ae3-97f3-4c95-9c5e-9dee6813cc0d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:55.122440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:55.123928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:55.124452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:55.125082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:55.132948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6391-8798-452c-ad50-0c173f447c93 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:55.204562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:55.206547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:55.233955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:55.234977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:55.235644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:55.236289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:55.248676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:55.328961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:55.348802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:55.351163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:55.472824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 130/264] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:55.479941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:55.518229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 135/265] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:55.526815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:55.672754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:55.710596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee7dd-6f24-4aa9-809b-6c6473ec9061 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:55.777554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857d387-f62c-4289-a2eb-daa4e4c3afcd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:55.796779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:55.797435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:55.797707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:55.797988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:55.800200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:55.801666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:55.803364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 131/266] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:55 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:55.809946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:55.810864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:55.856074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:55.856758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:55.857047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:55.857327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:55.863403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:55.865001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:55.867163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 136/267] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:55 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:55.876462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:56.030663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d0433-cb43-49d6-b137-261ae71221c4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:56.101146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c188324-7076-4776-b12e-28d66fdd545d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:56.126189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:56.126847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.127132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.127409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.128157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=c57f12edf1bf4f1099ed713e3e3ad12e) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:56.128468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:56.130053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a9531-995e-4980-bf19-3a7d90c98f61 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:56.144282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 132/268] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:55 2022] GET /identity/v3/role_assignments?user.id=c57f12edf1bf4f1099ed713e3e3ad12e&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:56.145695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60cdb-510c-4655-a059-12e3dfea3f83 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.145983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60cdb-510c-4655-a059-12e3dfea3f83 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.146260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60cdb-510c-4655-a059-12e3dfea3f83 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.147712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 133/269] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:56.155495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.155885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.156109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.156945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:56.157806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.158280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.158778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.159729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=c57f12edf1bf4f1099ed713e3e3ad12e) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:56.160324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:56.162085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ebb45-8f60-4aa0-9118-99c23be409a2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:56.170672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 137/270] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:55 2022] GET /identity/v3/role_assignments?user.id=c57f12edf1bf4f1099ed713e3e3ad12e&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:56.183307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3b4a-05cd-4496-894e-2c6ae0bdc19a None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.183937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3b4a-05cd-4496-894e-2c6ae0bdc19a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.184434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d3b4a-05cd-4496-894e-2c6ae0bdc19a None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.185785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:56.186454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 138/271] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:56.191666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:56.197146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.197803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.198905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:56.212704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:56.279183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:56.281616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:56.437258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 134/272] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:56.449974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:56.480551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 139/273] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:54:56.765402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:56.918334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:56.953574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db59482-ad75-4fc2-b4b1-967a965a8735 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:56.995427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:56.996071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:56.996350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:56.996623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.008731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-fc8a0467-9a40-4773-9f70-0b0a6fe66f40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:57.014196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:57.015112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 135/274] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:57.022184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bd852-164c-4c48-87be-4572fa08814f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.023079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bd852-164c-4c48-87be-4572fa08814f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.023079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bd852-164c-4c48-87be-4572fa08814f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.023079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 136/275] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:57.023549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bd852-164c-4c48-87be-4572fa08814f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.034014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.049092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99e642-0390-4792-b76b-8f7d788cdbd3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.091581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.092231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.092518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.092817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.093942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.095462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002fc16-4701-422e-a756-670cef76a66a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.097110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 140/276] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:57.099907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.100200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.100462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.119318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:57.129925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:57.140608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bd852-164c-4c48-87be-4572fa08814f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.165926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.176935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.176935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.176935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.177161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:57.177421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.178852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f74eac-8e8d-4ed3-810c-db234771e7ef None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.188033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 137/277] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:57.189390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6937422-4bc2-4a7c-82bb-3b2a39e12ec9 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.189676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6937422-4bc2-4a7c-82bb-3b2a39e12ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.189954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6937422-4bc2-4a7c-82bb-3b2a39e12ec9 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.191203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 138/278] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:54:57.191884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6937422-4bc2-4a7c-82bb-3b2a39e12ec9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.199033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 141/279] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:57.199033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.289773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.333515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5496-6f2b-4ea4-9c90-da68137529db None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.333750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6937422-4bc2-4a7c-82bb-3b2a39e12ec9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.369689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.370862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.371151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.371432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] PATH_INFO: `/v3/users/d8cd2a8f737b48ebae22a934bf8b8232` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.374567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.375712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.375712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.375876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.376551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.378018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.386422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] RBAC: Authorizing `identity:get_user(user_id=d8cd2a8f737b48ebae22a934bf8b8232)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.388972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dcfafa-36d4-470d-8338-617b10be0189 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.396756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 142/280] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/users/d8cd2a8f737b48ebae22a934bf8b8232 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:57.396756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.396756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.396756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.413702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:54:57.414935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:54:57.444665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.486082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 143/281] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:54:57.486983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.489195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 139/282] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:54:57 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:54:57.503757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.660219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60de04d-e18f-4f90-8792-302831ace7ef None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.674782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253bc47-b624-452a-b97c-5f83b4c04e80 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.692839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.722671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.722921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.723205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.723486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.728342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.728970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.729988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.730270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.732129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, user.id=d8cd2a8f737b48ebae22a934bf8b8232, scope.system=all) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:57.732129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.733323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.736914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:57.739382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 144/283] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/role_assignments?user.id=d8cd2a8f737b48ebae22a934bf8b8232&scope.system=all&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:57.743175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:57.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:54:57.745793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:57.748838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 140/284] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:57.754851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.787366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.793601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:57.909429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8644-25b0-4301-9f53-4caaff7e8e06 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.945547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac0937-c025-4692-95a0-cc8621aefc81 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:57.961833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.962687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.963385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.964231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:57.964931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:57.965247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:57.967149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:57.978239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 145/285] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:57.984522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:57.994244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:57.995548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:57.995803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:57.996102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.044305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:54:58.049760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:54:58.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:54:58.051472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 141/286] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:54:58.056978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.139520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:58.149924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:58.151606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee0a89-94b3-45bc-a723-2a12d811d6c3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:58.192059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:58.193007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:58.193532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:58.194052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] PATH_INFO: `/v3/users/8383814a592140cfbe9be293a27e4311` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.195705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] RBAC: Authorizing `identity:get_user(user_id=8383814a592140cfbe9be293a27e4311)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:58.197368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:58.200959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 146/287] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:57 2022] GET /identity/v3/users/8383814a592140cfbe9be293a27e4311 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:58.206756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.326402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69836977-fa66-41df-81c4-35f294395f97 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:58.336300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e8ebc-c9f4-4eb1-92d2-b05ca86d7c51 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:58.369408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:58.370956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:58.371576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:58.372050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.407030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:58.431442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:58.432918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:58.433445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:58.433896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.434774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:58.435357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:58.437811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:58.457846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 142/288] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:58.464711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.481351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:58.490647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:58.615906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 147/289] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:54:58.622654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.660422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6bbc-ab1f-4024-b992-7528588621cf None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:58.669617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:58.734121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:58.743176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:58.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:58.747049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:58.747049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] PATH_INFO: `/v3/users/157ff0631d874dbfaebc7ee30bba3242` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.747049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] RBAC: Authorizing `identity:get_user(user_id=157ff0631d874dbfaebc7ee30bba3242)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:58.748779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:58.750637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 143/290] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/users/157ff0631d874dbfaebc7ee30bba3242 => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:58.759136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.845084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e64a3-902c-4f04-885b-3e0a5327dfb2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:58.888963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:58.889613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:58.889897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:58.890176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:58.896743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:58.898246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:58.903407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 148/291] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:54:58.917852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:58.972856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31e1eb-99ce-4ffb-88b5-fea45995608a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:59.041449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:59.042877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:59.043232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:59.043516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:59.045487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:59.047017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:59.050117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 144/292] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:59.054444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:54:59.127894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa85004-62aa-4ce3-9e8e-4847357d7bd0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:59.144426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:59.180403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:54:59.208637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11333fa-e6e2-47e7-81e3-2d266c9bc77d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:54:59.219473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:59.220319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:59.220778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:59.221214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:59.222108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=8383814a592140cfbe9be293a27e4311) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:54:59.222570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:59.224239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44aca-060c-4714-9917-59cc6be7e1ef None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:59.245735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 149/293] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:54:58 2022] GET /identity/v3/role_assignments?user.id=8383814a592140cfbe9be293a27e4311&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:54:59.251840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:54:59.252717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:59.253196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:59.253577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/157ff0631d874dbfaebc7ee30bba3242/roles/035acef499d346fdaeadda8d604f717e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:59.255837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, user_id=157ff0631d874dbfaebc7ee30bba3242, role_id=035acef499d346fdaeadda8d604f717e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:54:59.257423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2f83-6325-43db-91e2-39ce0a55b595 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:54:59.349232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 145/294] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:54:59 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/users/157ff0631d874dbfaebc7ee30bba3242/roles/035acef499d346fdaeadda8d604f717e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:54:59.991861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd323aa-ce6a-4a21-9bb9-c1965b47b42b None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:54:59.992208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd323aa-ce6a-4a21-9bb9-c1965b47b42b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:54:59.992423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd323aa-ce6a-4a21-9bb9-c1965b47b42b None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:54:59.994060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 150/295] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:00.000269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b145-dcd0-4dcd-8e3e-0dae16ea1219 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.000556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b145-dcd0-4dcd-8e3e-0dae16ea1219 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.000834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b145-dcd0-4dcd-8e3e-0dae16ea1219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.005899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536def-fbad-4253-9ba8-0399ac5068a4 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.006188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536def-fbad-4253-9ba8-0399ac5068a4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.006464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536def-fbad-4253-9ba8-0399ac5068a4 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.007941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 151/296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:00.016904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21a49-289f-4151-a2c2-96e2dd3d826a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.017467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21a49-289f-4151-a2c2-96e2dd3d826a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.017940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21a49-289f-4151-a2c2-96e2dd3d826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.024541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-dbb7b145-dcd0-4dcd-8e3e-0dae16ea1219 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:00.028132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-dbb7b145-dcd0-4dcd-8e3e-0dae16ea1219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:00.047341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-95d21a49-289f-4151-a2c2-96e2dd3d826a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:00.049547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-95d21a49-289f-4151-a2c2-96e2dd3d826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:00.131048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 152/297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:00.132303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 146/298] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:00.140049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874bf56-eeb2-4868-b6e5-f9902dbacf4c None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.140564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874bf56-eeb2-4868-b6e5-f9902dbacf4c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.140989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874bf56-eeb2-4868-b6e5-f9902dbacf4c None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.142733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 153/299] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:00.151731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.151949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.152225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.172751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:00.174291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:00.203447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:00.253953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 147/300] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:00.418298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874bf56-eeb2-4868-b6e5-f9902dbacf4c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.428862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.530284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874bf56-eeb2-4868-b6e5-f9902dbacf4c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.547492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8bb61-1737-4980-827c-016dd35d6d1f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.552110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:00.566915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.567795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.567795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.568082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.579669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:00.584408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:00.585414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 154/301] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:00.591726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.592446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.592702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.593800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.593800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.606511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:00.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:00.612379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:00.613186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 148/302] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:00.613846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.640855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:00.717294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f5cc1-37fb-4e91-a711-3ed4359fad26 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.724750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17816997-d73a-4d05-b0a7-53c81ff4dd0d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.756256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.756986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.757273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.757552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.758706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:00.760158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:00.762253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.762448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 155/303] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:00.767163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.767163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.767409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.767484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.767484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:00.767484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:00.767484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:00.775749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 149/304] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:00.777393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.825358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:00.874521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:00.883126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0834d62-3794-48e9-af42-d2960bfe3b4d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.898037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fe7b0-38f5-4c7d-80be-dc894809c2d0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:00.923896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.924513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.924798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.925067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.925742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:00.926047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:00.927521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:00.937158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 156/305] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:00.937969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:00.944819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:00.946539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:00.946999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:00.947416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] PATH_INFO: `/v3/groups` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:00.948801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:00.950583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:01.032940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 150/306] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 26 05:55:00 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:01.032940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:01.058036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712e07c-8f86-4e44-acb4-aec987258d8b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:01.072029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:01.111150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:01.112436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.112932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.113882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] PATH_INFO: `/v3/users/d8cd2a8f737b48ebae22a934bf8b8232` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.115799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] RBAC: Authorizing `identity:get_user(user_id=d8cd2a8f737b48ebae22a934bf8b8232)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:01.118214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:01.121384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 157/307] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:00 2022] GET /identity/v3/users/d8cd2a8f737b48ebae22a934bf8b8232 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:01.128169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:01.183745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73677-207d-4445-b274-adf35fb816ab None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:01.235250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:01.235899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.236185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.236500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] PATH_INFO: `/v3/users/85220ee1729343989147f17714293ea5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.255983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] RBAC: Authorizing `identity:get_user(user_id=85220ee1729343989147f17714293ea5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:01.257470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:01.259397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 151/308] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:01 2022] GET /identity/v3/users/85220ee1729343989147f17714293ea5 => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:01.265686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:01.266399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a2c1-048a-4f99-8420-662115856b18 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:01.268947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:01.314725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:01.315324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.315606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.315944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] PATH_INFO: `/v3/system/users/d8cd2a8f737b48ebae22a934bf8b8232/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.317421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d8cd2a8f737b48ebae22a934bf8b8232, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:01.319365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a2910-5126-48a9-bd86-44af01256b54 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:01.384298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c0c62-9e57-4b0c-b086-5e4196492385 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:01.420235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:01.420846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.421129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.421403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.422133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, user.id=85220ee1729343989147f17714293ea5, scope.system=all) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:01.422444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:01.423934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4204e67-062a-46f7-abcb-c2aec6d961dc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:01.429453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 152/309] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 05:55:01 2022] GET /identity/v3/role_assignments?user.id=85220ee1729343989147f17714293ea5&scope.system=all&role.id=035acef499d346fdaeadda8d604f717e => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:01.450686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 158/310] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 26 05:55:01 2022] PUT /identity/v3/system/users/d8cd2a8f737b48ebae22a934bf8b8232/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:01.468307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849267a2-9da0-4715-af57-bf81cacb4bb0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.468649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849267a2-9da0-4715-af57-bf81cacb4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.468867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849267a2-9da0-4715-af57-bf81cacb4bb0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.470503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 153/311] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:01.476551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:01.476867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:01.477116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:01.499099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:01.500468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:01.545498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:01.589981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 159/312] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:01.875464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849267a2-9da0-4715-af57-bf81cacb4bb0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:01.951922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:01.989416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849267a2-9da0-4715-af57-bf81cacb4bb0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:02.029388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:02.030064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:02.030359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:02.030642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:02.043564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:02.048520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:02.049441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:02.050209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 154/313] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:02.059805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:02.176106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea58a5-eeeb-4419-9886-7042ec441600 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:02.214759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:02.215386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:02.215666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:02.216037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:02.216743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:02.217047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:02.218438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:02.227566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 160/314] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:02.234283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:02.350230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a499b69-55c7-47d0-8bc1-3026ea23b147 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:02.380439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:02.391038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:02.391763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:02.392043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:02.392316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] PATH_INFO: `/v3/users/157ff0631d874dbfaebc7ee30bba3242` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:02.393444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] RBAC: Authorizing `identity:get_user(user_id=157ff0631d874dbfaebc7ee30bba3242)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:02.395268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:02.406517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 155/315] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:02 2022] GET /identity/v3/users/157ff0631d874dbfaebc7ee30bba3242 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:02.406707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:02.451062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:02.516819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fe32b-3b47-456d-8f98-a0b847de0868 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:02.555414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:02.556088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:02.556375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:02.556656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:02.635213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:02.637961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:02.676177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:02.751098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 161/316] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:02.761345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:02.873644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df6210-a9e1-45d3-ad23-ea3b5e487c8f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:02.913615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:02.914255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:02.914533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:02.914833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:02.915988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:02.918256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff5c25-df22-46af-861a-00b48aa68e88 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:02.924965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 156/317] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:02 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:02.926363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:03.038439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299b4c8-a748-48ac-807e-8d0124f696e2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:03.075987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:03.076599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.076881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.077155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.077888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, scope.project.id=9f37696c758f4633b61e2f2baf5423bb, user.id=157ff0631d874dbfaebc7ee30bba3242) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:03.078197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:03.079857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee13e5e-e01f-4396-87a8-afc930cd87b0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:03.081814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:03.090726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 162/318] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:02 2022] GET /identity/v3/role_assignments?user.id=157ff0631d874dbfaebc7ee30bba3242&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=035acef499d346fdaeadda8d604f717e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:03.148865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a838-7dc3-4a54-b3d7-411ab32b2e38 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.149150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a838-7dc3-4a54-b3d7-411ab32b2e38 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.149435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76a838-7dc3-4a54-b3d7-411ab32b2e38 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.152037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 157/319] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:03.157988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a645-c516-45be-bc8d-37b3a62dc130 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.158281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a645-c516-45be-bc8d-37b3a62dc130 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.158568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a645-c516-45be-bc8d-37b3a62dc130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.183223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-aef5a645-c516-45be-bc8d-37b3a62dc130 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:03.184499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-aef5a645-c516-45be-bc8d-37b3a62dc130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:03.263448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 163/320] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:03.498077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7508-4103-42b3-93fa-8a9729950f87 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.498594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7508-4103-42b3-93fa-8a9729950f87 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.498894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7508-4103-42b3-93fa-8a9729950f87 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.501264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 158/321] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:03.506818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.507247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.507421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.527394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:03.528606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:03.531014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f424-1359-449c-b558-a6009b974e48 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.531298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f424-1359-449c-b558-a6009b974e48 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.531567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f424-1359-449c-b558-a6009b974e48 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.533029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 159/322] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:03.537769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.538047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.538314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.555987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:03.557263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:03.600257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 164/323] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:03.601047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:03.620757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:03.632404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 160/324] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:03.664022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:03.705517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343976a8-3e24-4cc8-bad8-31923dc4de57 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:03.740941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:03.741644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.741956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.742238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.743376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:03.744962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:03.746717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 165/325] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:03.753684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:03.877544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:03.916079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b827dbe-be28-4d0d-87d2-e7e1b68d975f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:03.951189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:03.951796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:03.952075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:03.952350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] PATH_INFO: `/v3/groups` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:03.952983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:03.954441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:03.987541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25914ebb-45c1-4b62-a1af-520bd8c74d07 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.012938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:04.021430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.022101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.022388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.028022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.033438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:04.038236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:04.039270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 166/326] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:04.043384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.052082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 161/327] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Aug 26 05:55:03 2022] POST /identity/v3/groups => generated 226 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:04.052775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.113990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:04.169877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010b38-faed-4c5f-ae08-e96ff5a1a667 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.171674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8500-a630-418b-baa8-1a3bd62b8c9a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.208148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.208906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.209227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.209559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.219125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.222766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.222766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.222766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.222766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:04.222766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.224004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:04.224163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:04.232577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:04.233531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:04.234531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 167/328] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:04.235537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 162/329] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:04.243236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.244092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.299035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:04.384485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782493b-6869-4027-a542-b39cd6bbcc96 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.387060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfcd7b-405a-4343-ba1f-e877af9d582e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.410405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:04.429851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.430906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.431402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.431843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.432883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:04.433338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.434956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.436495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.437171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.437458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.437738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] PATH_INFO: `/v3/users/85220ee1729343989147f17714293ea5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.440492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] RBAC: Authorizing `identity:get_user(user_id=85220ee1729343989147f17714293ea5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.440492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.442061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 163/330] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/users/85220ee1729343989147f17714293ea5 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:04.448496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.448815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 168/331] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:04.454885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.586975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f302f-7e2a-4cce-a6d0-16ced54b8b71 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.615351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56ffd2-f4db-4ee6-954a-cea2fc25c128 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.628191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:04.650595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.653104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.653843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.654327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] PATH_INFO: `/v3/system/users/85220ee1729343989147f17714293ea5/roles/035acef499d346fdaeadda8d604f717e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.658061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=85220ee1729343989147f17714293ea5, role_id=035acef499d346fdaeadda8d604f717e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.661646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4902d3b-ce6e-4c2f-b3a6-928d874908aa None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.674578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.675615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.675904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.676191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] PATH_INFO: `/v3/users/d8cd2a8f737b48ebae22a934bf8b8232` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.679959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] RBAC: Authorizing `identity:get_user(user_id=d8cd2a8f737b48ebae22a934bf8b8232)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.680825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.683531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 169/332] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/users/d8cd2a8f737b48ebae22a934bf8b8232 => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:04.689119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:04.787681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 164/333] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Aug 26 05:55:04 2022] PUT /identity/v3/system/users/85220ee1729343989147f17714293ea5/roles/035acef499d346fdaeadda8d604f717e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:55:04.808336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e778a7-6b3e-46a7-8703-9b31b4cbd35b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:04.852315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:04.852869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:04.853159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:04.853444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:04.854309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] RBAC: Adding query filter params (role.id=d14184d6cd4d4614b284d4c9394bf9ab, user.id=d8cd2a8f737b48ebae22a934bf8b8232, scope.system=all) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:04.854645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:04.856406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884b9d3-2576-485d-ba3c-8cb73c902bbc None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:04.872257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 170/334] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 05:55:04 2022] GET /identity/v3/role_assignments?user.id=d8cd2a8f737b48ebae22a934bf8b8232&scope.system=all&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:06.503026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae47d-d8a7-4222-97b5-1e291983b01a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:06.503539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae47d-d8a7-4222-97b5-1e291983b01a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:06.503539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae47d-d8a7-4222-97b5-1e291983b01a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:06.505268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 165/335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:06.510727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492a396-0ac0-4916-851e-a3f3c5424530 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:06.511271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492a396-0ac0-4916-851e-a3f3c5424530 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:06.511271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492a396-0ac0-4916-851e-a3f3c5424530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:06.539233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3492a396-0ac0-4916-851e-a3f3c5424530 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:06.540445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3492a396-0ac0-4916-851e-a3f3c5424530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:06.575014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:06.633311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 171/336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:07.053141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b33c7-6bd9-4a75-b53b-893d3b720aeb None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.053826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b33c7-6bd9-4a75-b53b-893d3b720aeb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.054343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b33c7-6bd9-4a75-b53b-893d3b720aeb None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.055936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 166/337] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:07.062663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b656ed-455e-4929-86b4-ce1440249384 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.063278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b656ed-455e-4929-86b4-ce1440249384 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.063772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b656ed-455e-4929-86b4-ce1440249384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.084558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-69b656ed-455e-4929-86b4-ce1440249384 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:07.086074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-69b656ed-455e-4929-86b4-ce1440249384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:07.157543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 172/338] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:07.199780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086ca98-a301-4501-a3ad-61e075e43a75 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.200393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086ca98-a301-4501-a3ad-61e075e43a75 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.200891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086ca98-a301-4501-a3ad-61e075e43a75 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.202429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 167/339] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:07.217077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086ca98-a301-4501-a3ad-61e075e43a75 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:07.221544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.222996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.223502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.261182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:07.263800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:07.313049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:07.391149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 173/340] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:07.422278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086ca98-a301-4501-a3ad-61e075e43a75 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:07.460569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:07.473961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:07.474648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.474937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.475219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.498392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:07.504955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:07.506258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:07.507114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:07.511397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:07.514731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 168/341] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:07.518885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:07.668283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b0f31-8a84-4f1d-aadb-f4ae3e40e0c6 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:07.705882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a57884-cdbc-4916-85c7-42317e630f4b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:07.715524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:07.734547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:07.738566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.739144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.739619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.759349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:07.764889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:07.765782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:07.766557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:07.769548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 174/342] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:07.775027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:07.787958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:07.789013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:07.789475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:07.789932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:07.791268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:07.795155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:07.797778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:07.822321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 169/343] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:07.828423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:07.891798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:07.929654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9bea3-3ea6-4e3f-b6b3-aef065f3e250 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.037688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:08.040277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.040577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.041086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.042838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:08.043409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:08.052054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf13dc-0842-48b9-8164-3c7c5adf2237 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:08.077824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 175/344] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:08.079679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824587-da0e-4368-9050-1fbc2b55034b None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.080188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824587-da0e-4368-9050-1fbc2b55034b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.080672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824587-da0e-4368-9050-1fbc2b55034b None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.083409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 176/345] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:08.088862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824587-da0e-4368-9050-1fbc2b55034b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:08.144348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cb2e9-9c9e-4c66-b47d-202b00a02c4d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.196839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:08.197983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.198378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.198852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.235332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824587-da0e-4368-9050-1fbc2b55034b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.254758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.284263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:08.285266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.285746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.286208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.304101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:08.304662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.306188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf362c7-0c09-4878-8d29-c6e518eb0990 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:08.378052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:08.379681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:08.452688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 170/346] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:55:08.452688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a935-d8b1-435c-8d94-6da781e6988e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.452965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a935-d8b1-435c-8d94-6da781e6988e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.453198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a935-d8b1-435c-8d94-6da781e6988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.477025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2c49a935-d8b1-435c-8d94-6da781e6988e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:08.478124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2c49a935-d8b1-435c-8d94-6da781e6988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:08.526393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 177/347] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:08.527360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:08.567183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.576962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 171/348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:08.578271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935a5-b85c-4bd2-a026-13acf930be00 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.578565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935a5-b85c-4bd2-a026-13acf930be00 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.578884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6935a5-b85c-4bd2-a026-13acf930be00 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.580843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 172/349] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:08.581101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6935a5-b85c-4bd2-a026-13acf930be00 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:08.591785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.669176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6899a-76fb-48a6-86ef-a0984c69eea7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.728802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:08.729968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.730589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.731058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.747564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6935a5-b85c-4bd2-a026-13acf930be00 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.756328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:08.764489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:08.765911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:08.766976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 178/350] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:08.768299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:08.824500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:08.825093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.825447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.825706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.827149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:08.830058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02905446-7464-4bc9-b86a-a5225a669737 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:08.830802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 173/351] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:08 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:08.832044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82956033-ca56-4637-9996-aec815e57e7c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:08.832720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82956033-ca56-4637-9996-aec815e57e7c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:08.832720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82956033-ca56-4637-9996-aec815e57e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:08.864948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-82956033-ca56-4637-9996-aec815e57e7c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:08.867533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-82956033-ca56-4637-9996-aec815e57e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:08.951963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c74d-cca5-4447-8c7e-e5125de7ecbb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:08.966148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.971397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:08.981888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 174/352] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:08.983675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82956033-ca56-4637-9996-aec815e57e7c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.012998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.013211 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.013456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.013701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.015027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.018706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 179/353] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:08 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:09.019553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.133283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82956033-ca56-4637-9996-aec815e57e7c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.144499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b402f33-5264-4f83-a126-38ce466a38bb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.182322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.183801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.184291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.184722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.185659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:09.186205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.188042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.188628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.189408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.189794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.190175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.197167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.199512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.201338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 175/354] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:09.202004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 180/355] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:09.206703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.207245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.270575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.298243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.353618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c66ff-6cec-4085-9a8b-3c74cc90cb8f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.356355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0dcd6-a87c-4209-9840-f80095753d7a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.404479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.405060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.405343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.405616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.406479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.407156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.407438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.407916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.409487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.411309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.413421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 181/356] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:09.414237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.420518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:09.426173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:09.427134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 05:55:09.427946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:09.428555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 176/357] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:09.429864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.551531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a9293-9633-4d9a-acb8-9175a1b09b3a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.563764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.586162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228ea02-a87f-4f50-956e-5293174d3cd0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.597447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.598068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.598351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.598846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.599462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:09.600431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.603092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.608833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.610170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 182/358] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:09.611050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.636234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.636466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.636830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.637136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.735837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:09.737025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:09.746842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469373a-af60-42ca-b2b0-86c66ca6f6b1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:09.796896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:09.797694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:09.797987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:09.798275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:09.812365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:09.817309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:09.818314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:09.819368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 183/359] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:09.820052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.857116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.893888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 177/360] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:09.895513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:09.922297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:09.955295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6c26-3b01-4cf1-904f-3b00ec949999 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.001679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.002334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.002779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.003125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.003930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:10.004312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.005766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796e013-f050-48f7-b911-3cd01828b5b3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.012517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 184/361] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.014168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7896a-ecdd-45e0-a1d5-fef6f24bb8b3 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.014461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7896a-ecdd-45e0-a1d5-fef6f24bb8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.014759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7896a-ecdd-45e0-a1d5-fef6f24bb8b3 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.016191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 185/362] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:10.018788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7896a-ecdd-45e0-a1d5-fef6f24bb8b3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.052113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad737eff-5e61-4039-8a8a-17f64b297712 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.111187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.113076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.115546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.116050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.117512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:10.117889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.120104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.130662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 178/363] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.131282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.171454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.182179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7896a-ecdd-45e0-a1d5-fef6f24bb8b3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.231625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.233124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.233691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.234173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.235030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:10.235499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.237961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e36df-5933-409f-be6d-2dc971ae3f50 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.251121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 186/364] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.252751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.253174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.253584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.262384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.279843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:10.281936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:10.285876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8642c2-dd36-4a55-9d7c-06458c5ebc85 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.339031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.339775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.340067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.340333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.341510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.344223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.346202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 179/365] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.347015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.388991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 187/366] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:10.389857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.477059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117a2ae-e96c-4107-b726-73dab7333e77 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.482721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.497915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b6be3-2410-4210-95d9-e84f65c4cb01 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.536087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.536634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.536923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.537206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.540185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.540881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.541348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.541593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] PATH_INFO: `/v3/users/85220ee1729343989147f17714293ea5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.547347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] RBAC: Authorizing `identity:get_user(user_id=85220ee1729343989147f17714293ea5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.548799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.550431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 180/367] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/users/85220ee1729343989147f17714293ea5 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.551364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.581494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.618015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.620134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.700107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df61e6-07b0-4a48-8fa1-51f3431ece30 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.714003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.731237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 188/368] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:10.732790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.754438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.758516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.759085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.759571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.762670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.764262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.766153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 181/369] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.769662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.873811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a34db2-a1f1-4418-9eeb-d1c725040b62 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.898596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:10.922633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.923294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.923584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.923864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:10.924605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] RBAC: Adding query filter params (role.id=035acef499d346fdaeadda8d604f717e, user.id=85220ee1729343989147f17714293ea5, scope.system=all) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:10.924952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:10.925524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa97210-04bf-4613-93e5-6fea8b32a58b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:10.926487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:10.943914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 189/370] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/role_assignments?user.id=85220ee1729343989147f17714293ea5&scope.system=all&role.id=035acef499d346fdaeadda8d604f717e => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:10.950690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:10.993190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:10.994961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:10.995672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:10.996231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.004180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.019640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:11.027609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:11.028572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:11.029369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 182/371] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:11.030635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.079219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860eea4-e532-46f2-8695-79139d62a006 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.123487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.125073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.125561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.126034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.128840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.131062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.133270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 190/372] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:10 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:11.134388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.264266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e5cd-f7b6-4ca8-a857-5502633b95bf None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.264458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c34604-faaf-4e04-b915-9ce82782046d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.285021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.309328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.310044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.310322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.310598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.317255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=9f37696c758f4633b61e2f2baf5423bb) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:11.317255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.317255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.317477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.317477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.317477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.317477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.317763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:11.317763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.319990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.325557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 183/373] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 260 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:11.330797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.332401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 191/374] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:11.337996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.353843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.465616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd90162-11b1-4978-85c7-82edd9fb57ac None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.512727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.513678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.514111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.514556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.536756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcd6ae-8ed8-4fc4-9240-dc7149de963a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.587428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.600152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.600874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.601158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.601434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.602705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.605184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.607310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 184/375] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:11.614516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.615452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.616989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.746690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 192/376] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:55:11.753871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.758293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:11.784189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d46cc-1dcf-413a-993f-f6c324135177 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.825378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.826001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.826286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.826550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.828273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=9f37696c758f4633b61e2f2baf5423bb) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:11.828273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.831321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.836064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 185/377] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:11.883064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:11.894821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61e485-7e87-492a-b7eb-0aca637b1fb3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:11.966738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:11.967698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:11.968131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:11.968555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] PATH_INFO: `/v3/projects/6316e285f13041a3ae4690b463c7dfa7` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:11.969978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] RBAC: Authorizing `identity:get_project(project_id=6316e285f13041a3ae4690b463c7dfa7)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:11.971670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6710e-5b61-4d48-9aeb-ef7e5b984716 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:11.974011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 193/378] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/projects/6316e285f13041a3ae4690b463c7dfa7 => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:11.979578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:12.134284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe3ba7-7d69-4beb-b825-4bca47a911e0 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.134593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe3ba7-7d69-4beb-b825-4bca47a911e0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.134870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe3ba7-7d69-4beb-b825-4bca47a911e0 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.137067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 194/379] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:55:12.142819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1fbe-16ba-4aaa-9799-51c6d25b8cd8 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.143114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1fbe-16ba-4aaa-9799-51c6d25b8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.143398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e1fbe-16ba-4aaa-9799-51c6d25b8cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.175633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5c2e1fbe-16ba-4aaa-9799-51c6d25b8cd8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:12.178257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5c2e1fbe-16ba-4aaa-9799-51c6d25b8cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:12.223308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a192cab-5e47-4f41-b921-e90f64242a1c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:12.293862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:12.296386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.296688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.297164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] PATH_INFO: `/v3/groups/4bf65c27a26749f0ae7a0779970055ee` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.310268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] RBAC: Authorizing `identity:get_group(group_id=4bf65c27a26749f0ae7a0779970055ee)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:12.311769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:12.315362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 186/380] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:11 2022] GET /identity/v3/groups/4bf65c27a26749f0ae7a0779970055ee => generated 229 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:12.320786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 195/381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:12.321670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:12.329289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:12.526950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312b16f-8881-4339-b1b9-1ceee02ba182 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:12.568787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:12.569797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.570279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.570783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.571727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] RBAC: Adding query filter params (group.id=4bf65c27a26749f0ae7a0779970055ee, role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=6316e285f13041a3ae4690b463c7dfa7) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:12.572872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:12.574746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356bd7a-0e48-4c31-9dce-814e56b90633 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:12.581552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 187/382] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:12 2022] GET /identity/v3/role_assignments?group.id=4bf65c27a26749f0ae7a0779970055ee&scope.project.id=6316e285f13041a3ae4690b463c7dfa7&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:12.692878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a6718-1ee7-432e-b368-4bc108eb94b3 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.693208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a6718-1ee7-432e-b368-4bc108eb94b3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.693478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a6718-1ee7-432e-b368-4bc108eb94b3 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.694984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 196/383] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:12.711801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.712434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.713478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.735244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:12.736888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:12.797705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a6718-1ee7-432e-b368-4bc108eb94b3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:12.842279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:12.851380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 188/384] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:12.903834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:12.939701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a6718-1ee7-432e-b368-4bc108eb94b3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:12.982494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:12.983184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:12.983470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:12.983748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:12.996583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:13.001770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:13.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:13.003492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 197/385] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:13.009501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.156692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c7418-5cc1-498f-a32b-2768f2cc2b1f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.206391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:13.207105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:13.207386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:13.207658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:13.208340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:13.208644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:13.210058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:13.222812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 189/386] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:13.229032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.343786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896391a-4526-4521-9f26-60c1d8f5bc6e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.372534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.382482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:13.383168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:13.383453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:13.383731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:13.390643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:13.390643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:13.390643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 198/387] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:13 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:13.390643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:13.398676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.507019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894212b-8061-4ff2-8089-e33dfb80db4a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.539006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:13.545547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:13.546696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:13.546989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:13.547269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:13.548659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:13.550328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:13.552174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 199/388] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:13 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:13.558473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.636099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:13.672715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73009dd9-3b7f-495f-9192-dd5de0ff2c50 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.710026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:13.710912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:13.711091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ccbc-b57c-453b-8df9-082611d3d554 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.711294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:13.711955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/groups/14a775443cda40f9ba3b55656eddf532/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:13.723666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, group_id=14a775443cda40f9ba3b55656eddf532, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:13.725141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:13.759075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:13.760289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:13.760578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:13.760853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:13.772145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:13.778008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:13.778913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:13.779844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 190/389] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:13.786864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:13.806359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 200/390] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 05:55:13 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/groups/14a775443cda40f9ba3b55656eddf532/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:55:13.959102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39aa4d-6044-4c88-b3e3-9507dbd1fd9f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:13.999046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:14.030891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:14.030891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.030891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.030891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.031622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:14.031622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:14.031911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:14.054443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:14.055073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 191/391] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:14.190484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccfa3b-b3c2-4547-8ed9-622c2e082fa3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:14.222118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:14.238517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:14.239184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.239465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.239739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.241596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:14.243219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebecee-b32c-46e3-8a9f-302eabe05f03 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:14.251088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 201/392] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:14 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:14.254318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:14.344456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abe7c-2d33-4ffd-b8de-21e0f4bf6a78 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.345063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abe7c-2d33-4ffd-b8de-21e0f4bf6a78 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.345540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33abe7c-2d33-4ffd-b8de-21e0f4bf6a78 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.348389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 202/393] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:14.352733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.353270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.354434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.374371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:14.376131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:14.458765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 203/394] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:14.492827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a3916-2312-4bee-b972-0665ce717e49 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:14.542300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:14.543020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.543303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.543578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.544739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:14.546181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:14.548213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 192/395] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:14 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:14.553829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:14.674605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab843a-2e92-4848-9217-9cb95a0a9be1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:14.704038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:14.714848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:14.715798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.716242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.717405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/groups/14a775443cda40f9ba3b55656eddf532/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.723457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] RBAC: Authorizing `identity:create_grant(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69, group_id=14a775443cda40f9ba3b55656eddf532, role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:14.725274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ceae50-40dc-49c3-802d-df6c6a434eed None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:14.800753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 204/396] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 05:55:14 2022] PUT /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69/groups/14a775443cda40f9ba3b55656eddf532/roles/c17cdb7e94834eee81063576196e3fd9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:14.821334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:14.848217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d09725-081b-410e-9fea-8134def9075b None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.849795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d09725-081b-410e-9fea-8134def9075b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.849795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d09725-081b-410e-9fea-8134def9075b None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.851786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 205/397] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:55:14.858928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772065c3-90a8-458e-87e7-392dd95c672f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:14.859561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772065c3-90a8-458e-87e7-392dd95c672f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:14.860060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772065c3-90a8-458e-87e7-392dd95c672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:14.881249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-772065c3-90a8-458e-87e7-392dd95c672f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:14.883077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-772065c3-90a8-458e-87e7-392dd95c672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:14.929130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:14.978313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 206/398] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:15.014571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4adc-9d11-4944-934f-ee5cb099cf75 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.015366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4adc-9d11-4944-934f-ee5cb099cf75 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.015961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d4adc-9d11-4944-934f-ee5cb099cf75 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.019874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 207/399] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:15.022452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4746f40-4b3a-4b04-8bb7-9bca91fe0e63 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.026699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.026991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.027272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.066942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:15.067655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.068289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:15.068423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.068637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.068921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.082112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:15.087070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:15.089515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:15.090296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 193/400] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:15.099769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.176863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 208/401] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 05:55:15.229960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fadb00-e031-406a-a449-431c6717ee00 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.242678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:15.284008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.285033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.285529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.286073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.286953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:15.287490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:15.289105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:15.295224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.305109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 194/402] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:15.313565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.383942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:15.493729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cac1c-1a8c-44f3-b8f4-2554dc9ed304 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.514442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a129664-11ae-4ef1-91e0-f7b9eb52b371 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.549921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.550574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.550903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.551181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.559919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.560878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.561358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.561826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.562377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:15.563429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:15.565899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:15.567546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:15.568721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:15.569519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 209/403] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:15.575845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.579047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 195/404] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:15.580546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.629038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:15.686756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d1da-258a-4b66-8946-392bed862a92 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.694495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:15.698328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a2534-6035-4f2c-9ff4-c471649af794 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.725059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.725702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.725987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.726263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.737955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:15.739204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.740067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.740545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.741036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.741912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:15.742418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:15.742779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:15.743634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:15.744877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 210/405] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:15.745550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.747640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:15.762778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 196/406] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:15.764104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.856347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a8542-8e0b-48f4-a258-bd1dcd06747f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.887296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6db1f6-dc1e-4b64-a785-11255be92e29 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:15.889429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:15.897808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.897808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.902035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.902035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.903165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:15.905359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:15.913796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 211/407] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:15.915308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.943278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:15.943901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:15.944186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:15.944458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:15.945534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:15.945752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:15.947746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:15.959092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 197/408] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:15.960400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:15.967648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:16.049077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ddb24-9fd8-4582-aa53-d3670474ffba None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.080765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3b33-9efa-48a3-90cb-c1b618c088a8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.088591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.089227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.090464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.090464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.092370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.093763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.095578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 212/409] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:15 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:16.100796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.131084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.132201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.132682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.133159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/groups/14a775443cda40f9ba3b55656eddf532/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.136533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, group_id=14a775443cda40f9ba3b55656eddf532, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.138281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.241954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 198/410] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 05:55:15 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/groups/14a775443cda40f9ba3b55656eddf532/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:16.243455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.247573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426be3e-07c2-4259-85b5-c2543a5c6cf4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.254076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:16.295615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:16.334448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.335660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.336506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.337011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] PATH_INFO: `/v3/groups/4bf65c27a26749f0ae7a0779970055ee` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.338575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] RBAC: Authorizing `identity:get_group(group_id=4bf65c27a26749f0ae7a0779970055ee)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.340860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4e1e7-e1bd-40e1-b22b-3286f593ffdd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.343965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 213/411] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:16 2022] GET /identity/v3/groups/4bf65c27a26749f0ae7a0779970055ee => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:16.345791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188877-e9d9-4877-9e6b-5c6299545513 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.346436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188877-e9d9-4877-9e6b-5c6299545513 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.347362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188877-e9d9-4877-9e6b-5c6299545513 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.349713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 214/412] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:16.350811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab188877-e9d9-4877-9e6b-5c6299545513 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.405525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdbf0e-b895-456e-b104-8a4ba237ba38 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.458669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.458669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.458669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.459022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.460132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.461696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f754b14-a921-4c43-bc77-c1e622a01158 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.463915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 199/413] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:16 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:16.465202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.465486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.465766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.487986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:16.489256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:16.523228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab188877-e9d9-4877-9e6b-5c6299545513 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.573893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 200/414] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:16.575094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.584986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.585632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.585975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.586254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] PATH_INFO: `/v3/projects/6316e285f13041a3ae4690b463c7dfa7` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.587580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] RBAC: Authorizing `identity:get_project(project_id=6316e285f13041a3ae4690b463c7dfa7)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.589806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.597930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 215/415] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:16 2022] GET /identity/v3/projects/6316e285f13041a3ae4690b463c7dfa7 => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:16.604828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.633339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:16.696234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:16.785594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dec86-12ae-4df3-952c-34a7138fb23c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.801073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f3f79-e5a7-4e58-be5d-44a9ea9e4f10 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:16.832401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.833070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.833351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.833626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] PATH_INFO: `/v3/projects/6316e285f13041a3ae4690b463c7dfa7/groups/4bf65c27a26749f0ae7a0779970055ee/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.835602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] RBAC: Authorizing `identity:create_grant(project_id=6316e285f13041a3ae4690b463c7dfa7, group_id=4bf65c27a26749f0ae7a0779970055ee, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.838474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.861541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:16.862589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:16.863093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:16.863644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb/groups/14a775443cda40f9ba3b55656eddf532/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:16.866127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f37696c758f4633b61e2f2baf5423bb, group_id=14a775443cda40f9ba3b55656eddf532, role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:16.867797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:16.907129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 216/416] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 05:55:16 2022] PUT /identity/v3/projects/6316e285f13041a3ae4690b463c7dfa7/groups/4bf65c27a26749f0ae7a0779970055ee/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:55:16.908681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:16.945187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 201/417] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Aug 26 05:55:16 2022] PUT /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb/groups/14a775443cda40f9ba3b55656eddf532/roles/c17cdb7e94834eee81063576196e3fd9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:17.035224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:17.040250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df83f-7ab0-4fc7-8201-58965a3cf637 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:17.084055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:17.084814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.085100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.085376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.108093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d0a048fe-5234-4c69-85dc-2aac2bf8c1ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:17.112963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:17.113873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:17.114698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 217/418] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:17.123060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:17.140767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75aaf-d4ad-4e02-b273-74565e75f619 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.141078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75aaf-d4ad-4e02-b273-74565e75f619 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.141323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd75aaf-d4ad-4e02-b273-74565e75f619 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.142967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 218/419] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:17.148513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.148791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.149073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.171941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:17.173580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:17.247717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:17.264093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 219/420] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:17.270283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239222a-a6b0-405b-af7e-dd47ea09d995 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:17.311219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:17.312274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.312775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.313222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.314134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:17.314608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:17.316229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:17.326006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 202/421] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:17.331607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:17.476013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:17.490395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a6b19-e5cd-4a19-ad69-286ee529824d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:17.545292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:17.545927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.546214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.546491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.619730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:17.645288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:17.646800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:17.770102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:17.795664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 220/422] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:17.801130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:17.812081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd72d1-7cac-438d-95db-90242f8c6eb2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:17.862990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:17.864139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:17.865448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:17.866079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:17.887421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:17.894328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:17.895360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:17.896160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 203/423] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:17.900460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:17.908697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:18.037587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dd4ff-24bf-403f-8844-68d6cf5edbc3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.078699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1768b9-dff3-4fd9-9dac-fc8032adcd21 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.085846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.086719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.087149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.088115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.089008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:18.089512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.095256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.115598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.115598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.116043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.116043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.116996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 204/424] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:18.117159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.119864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.121029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 221/425] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:17 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:18.122015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:18.126812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:18.201801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:18.228213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:18.246200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c64426-d12f-46af-9794-19d177be5d2e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.286017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384799dc-9631-4337-a571-e84980815126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.286947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384799dc-9631-4337-a571-e84980815126 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.287381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384799dc-9631-4337-a571-e84980815126 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.287865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384799dc-9631-4337-a571-e84980815126 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.289260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384799dc-9631-4337-a571-e84980815126 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.291595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384799dc-9631-4337-a571-e84980815126 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.293549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 222/426] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:18 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:18.299489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384799dc-9631-4337-a571-e84980815126 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:18.313746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89bd8b-06f6-4991-b296-b2a9155d4f9e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.411366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.412006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.412295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.412577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.439391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384799dc-9631-4337-a571-e84980815126 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.478845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:18.501758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.502647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.503080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.503524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.504426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:18.506056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.507700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.520710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 223/427] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:18 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:18.585200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.586946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858436a-960d-42d9-9220-d5cf27ca5c68 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.683510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:18.751877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 205/428] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:55:18.757745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:18.851829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd3cc-a34c-4c8b-8b9b-2201d1cb5425 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.852294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd3cc-a34c-4c8b-8b9b-2201d1cb5425 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.852661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd3cc-a34c-4c8b-8b9b-2201d1cb5425 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.854954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 206/429] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:55:18.866051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.866375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.866596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.899279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:18.899279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:18.907668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb85a8-6069-41a3-b89d-bde733c8fe80 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:18.951090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:18.951751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:18.952039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:18.952318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] PATH_INFO: `/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:18.953729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] RBAC: Authorizing `identity:get_project(project_id=3940f6b6bcef4ac9a27e6a7e95fa3f69)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:18.955318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:18.957018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 224/430] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:18 2022] GET /identity/v3/projects/3940f6b6bcef4ac9a27e6a7e95fa3f69 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:18.966434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:18.994746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 207/431] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:19.051042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:19.092542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d267ae-ad5b-4e0a-ada3-6663ef0f35bd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.132437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.133005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.133297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.133578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.134865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:19.136469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:19.138407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 225/432] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:18 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:19.144219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:19.255476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:19.263104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ddbc7-20c3-4e71-b24b-351b8718815d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.303078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.303696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.304031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.304311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.305552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:19.307642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:19.307959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:19.309415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4fbe1-72e0-4000-bcea-db3b053b0964 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:19.322648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 208/433] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:19 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=3940f6b6bcef4ac9a27e6a7e95fa3f69&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:19.388327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8ea1-a359-4574-bb94-21d9bf9e2a6f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.388513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8ea1-a359-4574-bb94-21d9bf9e2a6f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.388784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a8ea1-a359-4574-bb94-21d9bf9e2a6f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.390447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 209/434] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:55:19.396521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d59b7-f64d-4935-8d6c-e57ad35d6955 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.396818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d59b7-f64d-4935-8d6c-e57ad35d6955 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.397092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d59b7-f64d-4935-8d6c-e57ad35d6955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.418255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-206d59b7-f64d-4935-8d6c-e57ad35d6955 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:19.419523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-206d59b7-f64d-4935-8d6c-e57ad35d6955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:19.438255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470ece5-e824-4e0a-8ad4-4c5336916038 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.467061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:19.480048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.480702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.480988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.481271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.495003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:19.501451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:19.502379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:19.503207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 226/435] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:19.506177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 210/436] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:19.509087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:19.523453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1b0b-8b0f-48cd-9d49-72f11dfe70d0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.523783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1b0b-8b0f-48cd-9d49-72f11dfe70d0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.524001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a1b0b-8b0f-48cd-9d49-72f11dfe70d0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.525805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 211/437] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:19.532003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.532296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.532556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.555378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:19.557670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:19.648790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 212/438] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:19.653375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b60bb5-c94e-4b54-bcee-10a7fcef0afe None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.702250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.702901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.703188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.703468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.704148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] RBAC: Adding query filter params (name=member) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:19.704457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:19.705857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:19.716216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 227/439] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:55:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:19.723170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:19.750151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:19.809379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:19.860862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcbe8f-03c7-4ce9-a488-63cac468d880 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.877117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:19.925622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.926267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.926666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.926831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:19.951053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e798-0fec-4f97-bb37-931807c3e371 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:19.991449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:19.992097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:19.992386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:19.992666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.004555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:20.009814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:20.010920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:20.011760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 228/440] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:20.015502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.018118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.019667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.041730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.121311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60076a4-5125-44e6-af9b-d792e71278dd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.128577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.132199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 213/441] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:20.133048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.159561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.160337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.160920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.161198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.172092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:20.177022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:20.178072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 229/442] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:20.179246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.240464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f90f1-c301-4996-bb7a-9bee5446da4c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.261126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.281883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.285374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.285374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.285374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.285374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:20.285374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.285832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5097-ed3b-4652-bc48-fac7f1f80b84 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.285932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.299335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 214/443] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.299335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.321423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.322122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.322398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.322705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.323833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.325281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.326928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 230/444] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.327774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.363153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.409513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c68b2-cbcd-4be3-800c-9dcddbdb8820 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.430969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59160f-b419-4330-8199-1475aadb80c4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.445677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.446291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.446569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.446902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.447587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:20.447891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.449279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.458034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 215/445] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.458034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.480690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.481310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.481613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.481892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.512169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.551846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.553336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.567636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259d0a9-a131-4b9e-91f9-7169075cfcd2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.586648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.601787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.602413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.602729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.603004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.605123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.606551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.608352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 216/446] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.608887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.653272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 231/447] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:20.654109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.713842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cda57-c4f1-4f8c-96cd-255744253d77 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.726558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.747384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.747996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.748280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.748557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.756299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228c015-d2e6-4b63-ae48-bd168e2a57fd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.791136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.791729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=d14184d6cd4d4614b284d4c9394bf9ab, scope.project.id=9f37696c758f4633b61e2f2baf5423bb) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.797351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.801085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 232/448] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=d14184d6cd4d4614b284d4c9394bf9ab => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.801872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.819375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.820883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.927853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece32c74-6ab2-4c8c-83d5-29747124c311 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:20.941838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 217/449] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:20.948893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:20.967659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:20.978001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:20.979422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:20.979648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:20.982550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] PATH_INFO: `/v3/projects/9f37696c758f4633b61e2f2baf5423bb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:20.982550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] RBAC: Authorizing `identity:get_project(project_id=9f37696c758f4633b61e2f2baf5423bb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:20.982550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:20.984367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 233/450] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/projects/9f37696c758f4633b61e2f2baf5423bb => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:20.989750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:21.078669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e91012-c15d-49a9-8f05-efe88ed368c5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:21.079652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:21.122239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:21.122583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9279e2e-9172-4376-be58-993536357c2d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:21.123056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:21.123234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:21.123490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] PATH_INFO: `/v3/projects/6316e285f13041a3ae4690b463c7dfa7` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:21.125588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] RBAC: Authorizing `identity:get_project(project_id=6316e285f13041a3ae4690b463c7dfa7)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:21.127173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:21.129009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 218/451] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/projects/6316e285f13041a3ae4690b463c7dfa7 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:21.135305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:21.168055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:21.168693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:21.168978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:21.169258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] PATH_INFO: `/v3/groups/14a775443cda40f9ba3b55656eddf532` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:21.170775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] RBAC: Authorizing `identity:get_group(group_id=14a775443cda40f9ba3b55656eddf532)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:21.172240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:21.174240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 234/452] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:20 2022] GET /identity/v3/groups/14a775443cda40f9ba3b55656eddf532 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:21.180006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:21.220329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:21.238496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad402d-6ef4-4859-be4c-c5e378bcc81b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:21.273084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:21.273694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:21.273971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:21.274240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] PATH_INFO: `/v3/groups/4bf65c27a26749f0ae7a0779970055ee` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:21.275667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] RBAC: Authorizing `identity:get_group(group_id=4bf65c27a26749f0ae7a0779970055ee)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:21.277231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:21.278809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 219/453] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Aug 26 05:55:21 2022] GET /identity/v3/groups/4bf65c27a26749f0ae7a0779970055ee => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:21.281072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea373fc-2a8c-4a06-8983-c679ddc9810e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:21.284625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:21.317298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:21.317921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:21.318207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:21.318509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:21.319272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] RBAC: Adding query filter params (group.id=14a775443cda40f9ba3b55656eddf532, role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=9f37696c758f4633b61e2f2baf5423bb) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:21.319592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:21.321034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cfb88-48b0-4429-b5d5-da6516c3eb45 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:21.323054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:21.328145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 235/454] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:21 2022] GET /identity/v3/role_assignments?group.id=14a775443cda40f9ba3b55656eddf532&scope.project.id=9f37696c758f4633b61e2f2baf5423bb&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:21.386731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a88116-1d2e-4ad7-801e-5cae9349d5d4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:21.421971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:21.422585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:21.422890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:21.423162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:21.423895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] RBAC: Adding query filter params (group.id=4bf65c27a26749f0ae7a0779970055ee, role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=6316e285f13041a3ae4690b463c7dfa7) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:21.424201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:21.425649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add295-689c-4ae6-bbc5-be74d8ec7b93 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:21.432794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 220/455] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Aug 26 05:55:21 2022] GET /identity/v3/role_assignments?group.id=4bf65c27a26749f0ae7a0779970055ee&scope.project.id=6316e285f13041a3ae4690b463c7dfa7&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:24.179325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebdbcc-49d2-4b48-8188-9ab99c5d917e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.179884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebdbcc-49d2-4b48-8188-9ab99c5d917e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.179884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebdbcc-49d2-4b48-8188-9ab99c5d917e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.182485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 236/456] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:24.189384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76eea-055f-4243-a93a-eb29f04dee77 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.189875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76eea-055f-4243-a93a-eb29f04dee77 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.190360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe76eea-055f-4243-a93a-eb29f04dee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.197705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220625-b904-441c-90d4-c1a99e66723a None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.198144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220625-b904-441c-90d4-c1a99e66723a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.198426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd220625-b904-441c-90d4-c1a99e66723a None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.200018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 237/457] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:24.206549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a3e-58ee-4b7d-99aa-e5df86a600d1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.206549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a3e-58ee-4b7d-99aa-e5df86a600d1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.206549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94a3e-58ee-4b7d-99aa-e5df86a600d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.224178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-afe76eea-055f-4243-a93a-eb29f04dee77 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:24.225731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-afe76eea-055f-4243-a93a-eb29f04dee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:24.231698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-27c94a3e-58ee-4b7d-99aa-e5df86a600d1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:24.233266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-27c94a3e-58ee-4b7d-99aa-e5df86a600d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:24.239158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:24.343809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 238/458] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:24.367360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 221/459] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:24.594454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ec43-edb6-4464-ab52-954ba086d217 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.594454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ec43-edb6-4464-ab52-954ba086d217 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.594454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ec43-edb6-4464-ab52-954ba086d217 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.595312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 239/460] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:24.601055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.601342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.601625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.631643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:24.632851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:24.680365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280ec43-edb6-4464-ab52-954ba086d217 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:24.728836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 222/461] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:24.818307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280ec43-edb6-4464-ab52-954ba086d217 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:24.826609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:24.855033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:24.864313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:24.864954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.866057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.866057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.879284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:24.881220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-7ae4b512-8632-4216-a158-03d54ed2c0e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:24.905463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:24.906396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:24.907437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:24.907437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:24.907437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:24.907620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 240/462] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:24.908870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a04b-0155-4d68-8aca-e2241f59ab95 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:24.909165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a04b-0155-4d68-8aca-e2241f59ab95 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:24.910651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a04b-0155-4d68-8aca-e2241f59ab95 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:24.911032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 241/463] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:24.914067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a04b-0155-4d68-8aca-e2241f59ab95 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:24.959217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:24.979627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4bb03-bdf4-481a-861b-bfcd111ca06f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.039089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.040332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.041825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.042405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.044396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a04b-0155-4d68-8aca-e2241f59ab95 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.062445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.065329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-fdc47d8d-fc24-4327-a2b8-7f43f3902969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:25.085541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:25.086488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:25.088520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:25.088520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:25.088520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:25.088520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 223/464] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:25.090998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.091903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.092331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.092763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.093765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:25.093921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.094209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.094453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.095013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.096502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.112328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 242/465] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:25.115512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:25.116718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:25.117770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.170369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.198690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 224/466] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:25.199483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.259790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de8d1f-50df-4443-a467-b5ea3b16d9df None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.268420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.302122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.302888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.303177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.303475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.315432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.316944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.329564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf836d-5f1b-4e05-9c39-861c64bbff57 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:25.334779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:25.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:25.336663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:25.336663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:25.336663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:25.336663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 243/467] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:25.336890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.379628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.380503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.380503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.380864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.381485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:25.381743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.383197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90515178-ed27-4e7d-9edf-0aa2b6e688a2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 225/468] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65537b9-7d46-4573-b396-5b3be0d47ba0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65537b9-7d46-4573-b396-5b3be0d47ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65537b9-7d46-4573-b396-5b3be0d47ba0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 226/469] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:25.402311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65537b9-7d46-4573-b396-5b3be0d47ba0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.456746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.466953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e134f-234e-4e30-ab2b-b53776af88dd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.507190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.507805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.508092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.508382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.509028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.511159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d88d58-a96b-4423-8fce-2c7f345909da None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.518422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65537b9-7d46-4573-b396-5b3be0d47ba0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.533183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.552496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.553089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.553369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.553638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.554268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:25.554566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.555981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.567568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 227/470] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:25.568359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.631453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 244/471] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:25 2022] POST /identity/v3/users => generated 246 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:25.632635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.632950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.633286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.669370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:25.671336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:25.690371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6ef2-368d-4a21-94d7-c39d84554c96 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.732474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.733104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.733382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.733657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.734543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.735889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:25.758280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 245/472] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:25.761012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.849210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.882059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 228/473] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:25 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:25.883462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:25.902345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050a085-f6ce-473b-a9fa-3e228c9973bb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:25.949872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:25.964393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:25.965045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:25.965326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:25.965600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:25.982208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:25.983822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:26.005220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:26.006195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:26.007475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:26.007475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:26.007475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:26.007475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 246/474] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:26.017314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:26.106974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874558a-5a49-484f-9ca9-752de0ea2ef2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.151729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.152609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.153051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.153471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.154280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:26.156659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.190028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7000-587f-43a1-a14a-e382c9f9d399 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.226900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:26.233546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.234168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.234448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.234755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.235700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:26.235806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:26.237363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.250676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 229/475] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:25 2022] POST /identity/v3/users => generated 248 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:26.250676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:26.266745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 247/476] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:26.268524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:26.379821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:26.404002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6a8f3-d593-49e3-83b0-1fdac0af8528 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.447396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.448955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.449375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.449816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.450715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:26.452385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.494155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bdf63-ce8c-48ff-84b3-93036cdba3ca None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.499353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.sql.core [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 05:55:26.499353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 05:55:26.499353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [parameters: {'id': 'b415009acb3d4e2c9ecf0f080a60cb24', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 05:55:26.499353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81762) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result.read() Aug 26 05:55:26.523972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:55:26.531271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application yield resource Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise self.value Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application session.commit() Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 05:55:26.532187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.session.flush() Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application compat.raise_( Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 05:55:26.533445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:55:26.534515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result.read() Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:55:26.535732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [parameters: {'id': 'b415009acb3d4e2c9ecf0f080a60cb24', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:26.536763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 05:55:26.537760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.538835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 230/477] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 05:55:26 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:26.539312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:26.553456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.554464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.555420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.556493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.557908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:26.560663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.652345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 248/478] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:26 2022] POST /identity/v3/users => generated 249 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:26.705755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f40b0-c7b9-413b-8710-e3289fd69992 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.747179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.747841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.748120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.748392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.760993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5372fe99-542a-41ab-8334-0d91c60949fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:26.766667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:26.767629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:26.768872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 231/479] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:55:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:26.774064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:26.832879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:26.923746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40672409-5771-4759-aa42-a1bdd130e313 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:26.961364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:26.961921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:26.962238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:26.963264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:26.963968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:26.964273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:26.965662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676af8b-fad8-4d28-bb9c-96b71fbf58bc None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:26.975148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 249/480] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:28.045315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064116d0-2f4b-468e-9e3f-8017fb5983e3 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.045817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064116d0-2f4b-468e-9e3f-8017fb5983e3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.045930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064116d0-2f4b-468e-9e3f-8017fb5983e3 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.047454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 232/481] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:28.055778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7874-78d2-45cc-9daf-93de63c86186 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.056858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7874-78d2-45cc-9daf-93de63c86186 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.056858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7874-78d2-45cc-9daf-93de63c86186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.083881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-812b7874-78d2-45cc-9daf-93de63c86186 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:28.085097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-812b7874-78d2-45cc-9daf-93de63c86186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:28.184838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 250/482] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:28.330144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e43f-3756-4ee2-9a46-9a1f59ab4b7c None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.330464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e43f-3756-4ee2-9a46-9a1f59ab4b7c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.330727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155e43f-3756-4ee2-9a46-9a1f59ab4b7c None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.332229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 233/483] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:28.338083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb8c9-27fd-460e-b349-a900a71ffe73 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.338366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb8c9-27fd-460e-b349-a900a71ffe73 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.338666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bb8c9-27fd-460e-b349-a900a71ffe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.363643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-306bb8c9-27fd-460e-b349-a900a71ffe73 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:28.364859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-306bb8c9-27fd-460e-b349-a900a71ffe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:28.408913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:28.468444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 251/484] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:28.652228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e43f-3756-4ee2-9a46-9a1f59ab4b7c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:28.666856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:28.704666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c9dd8-e432-42d3-a077-98309928defa None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.705016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c9dd8-e432-42d3-a077-98309928defa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.705221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c9dd8-e432-42d3-a077-98309928defa None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.706714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 252/485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:28.712606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.713399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.713892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.743229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:28.744636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:28.818651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155e43f-3756-4ee2-9a46-9a1f59ab4b7c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:28.858355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:28.859018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:28.859679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:28.859958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:28.869604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 253/486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:28.875734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:28.877714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:28.886787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:28.887684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:28.888456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 234/487] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:28.898669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.044537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.077860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.084185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db35b4b-feeb-40d9-9034-54f3b41b0ca4 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.114392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044413bb-633f-4d87-abf5-c3f987c2d67e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.152244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.153030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.153317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.153592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.174879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:29.181148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:29.182800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:29.184447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 254/488] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:29.195334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.196042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.196956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.197308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.198217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:29.198583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.199070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.200032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.210023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 235/489] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:29.210844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.328049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11170d1-40d3-40db-94c3-318412f83f8d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.332989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199badb8-8d26-401a-91df-70de8f2b7bda None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.364947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.367731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.368391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.368678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.369098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.369811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:29.370208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.371033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.371548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.371710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.371957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.372235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.382859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 255/490] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:29.382859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.383694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-14196e72-9fd5-4aa6-9f17-e76cf88cc335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:29.388713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:29.389811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:29.390652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 236/491] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:29.391353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae157d4-6ccc-4792-8425-749f1c7c0c20 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.391691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae157d4-6ccc-4792-8425-749f1c7c0c20 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.391968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae157d4-6ccc-4792-8425-749f1c7c0c20 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.393203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 237/492] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:29.393897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae157d4-6ccc-4792-8425-749f1c7c0c20 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.412040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.496665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b2d1ac-4193-47c3-8670-c95bf0cc8a9f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.511076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae157d4-6ccc-4792-8425-749f1c7c0c20 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.534655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.535300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.535589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.535869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.546514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.548016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.549043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.549678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.549959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.550232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.561266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:29.562708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:29.563538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:29.564354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:29.564354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:29.564354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:29.564511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.564576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 256/493] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:29.564647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-89b2ca82-26dc-4c14-8939-8bc96a04c0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:29.577696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:29.578788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 238/494] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418959-824e-45c9-ba2d-8ac7481daf67 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418959-824e-45c9-ba2d-8ac7481daf67 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.585248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c418959-824e-45c9-ba2d-8ac7481daf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.599447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9c418959-824e-45c9-ba2d-8ac7481daf67 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:29.600669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9c418959-824e-45c9-ba2d-8ac7481daf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:29.640981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.651682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.668837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 239/495] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:29.670117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95b1a-df4d-4126-beee-61288ba55ac4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.670406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95b1a-df4d-4126-beee-61288ba55ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.670717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95b1a-df4d-4126-beee-61288ba55ac4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.671990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 240/496] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:29.672692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a95b1a-df4d-4126-beee-61288ba55ac4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.684646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ba871-065d-4a0e-8fb9-3c53a45a96dc None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.725102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.725868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.726195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.726477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.727412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:29.727889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.730430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.743802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 257/497] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:29.743802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.796663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a95b1a-df4d-4126-beee-61288ba55ac4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.838690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.839367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.839649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.839983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.840636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:29.840938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.842670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a3887-e6c4-4979-921d-088fc2a3780f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.851252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce0e98-719c-4088-b6c4-8ee7fa1ea941 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:29.860874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 241/498] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:29.860874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.860874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.860874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.877491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:29.878955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:29.892136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:29.892821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:29.893111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:29.893389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:29.894047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:29.894352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:29.895847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:29.897377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.917686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 258/499] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:29.918302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:29.921409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:29.966346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 242/500] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:29.967239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.053348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5715b-e762-42b6-9619-4ea1e093dce7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.084979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55930c-54dd-4232-8de2-88c41cf0aa44 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.090191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.090818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.091197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.091498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.104609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.106215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:30.126538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.127528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:30.128337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:30.128337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:30.128337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:30.128337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 259/501] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.128555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.135513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.136263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.136558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.137079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.144884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.146355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:30.161171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.162304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 243/502] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:55:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.163258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.190134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.191982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.259462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22c34d-e45b-41bd-9d1b-a657584f4502 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.271991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13020e22-572c-424f-b274-69ec9ea9fd72 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.297641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.298304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.298593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.298902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.306833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.308183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.308435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.308813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.309089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.309360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.319364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:30.321281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.322142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 260/503] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.327852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:30.328530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.329319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 244/504] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.330084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.420518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.439094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f9981-3733-4708-b77a-b9ff964e72f2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.440078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b5e3c-40b7-4353-9567-f694d8696b5d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.448528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.476077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.476796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.477075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.480460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.480460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:30.480460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.480749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.480838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.480924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.480924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.480924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.481479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:30.481788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.483230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.499830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 261/505] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:30.500721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.523165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 245/506] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:30.523165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.616924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be149cbd-2c84-42cc-8abc-b06a39c67542 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.626651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda8c3d-990b-40cf-9347-eeebb499a307 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.656655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.657283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.657565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.657845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.664486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.665093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.665860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.665860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.666449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:30.666789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.668232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.669310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.670853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.682509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:30.685570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.686785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:30.687852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:30.687852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:30.687852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:30.688011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 246/507] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:30.688079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 262/508] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.688079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.689068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.710027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.792318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7e3f5-a955-42fe-9a28-587dc99580ba None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.793288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73692bc1-c683-4114-9de4-802b563686c7 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.828542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.829143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.829419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.829614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.829721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.830243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.830334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:30.830525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.830641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:30.831979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.838712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:30.840171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:30.841138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 247/509] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:30.841955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:30.857178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:30.858283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 263/510] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:30.859522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:30.909169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.948323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881726f-d6b3-4740-bf53-0b7c8d007971 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.948517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:30.970456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717cd9a-a95e-474f-befe-b29299491ce7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:30.982022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:30.982649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:30.982932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:30.983201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.005058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.005670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.005955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.006238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.050550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.052028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.075103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.076596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.112165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.152664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 248/511] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:31.153509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.154100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.178782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 264/512] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:31.179625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.255296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1886db-7adc-4e53-992b-6d21636623be None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.280646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ba3b6-4e4d-4e39-8854-0b0436afedc5 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.289385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.289987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.290892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.291440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.291864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:31.292163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.293558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.304659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 249/513] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:31.305472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.318831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.319459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.319740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.320012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.333924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.335403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.337129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:31.351192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:31.358335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:31.359224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:31.359224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.359224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:31.359224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 265/514] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:31.359224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.383141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.413592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ee9a1-84a0-45a4-92d0-4cfce27e3e5d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.448648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.449265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.449544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.449812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.450546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:31.450994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.452457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.456729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eba3a-97f7-4d7e-97bd-26865cf2f216 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.471043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 250/515] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:31.471902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.495219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.495854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.496135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.496414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.506666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.508109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:31.523298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:31.524365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:31.526455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:31.526455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.526455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:31.526455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 266/516] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:31.526750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.558011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.577048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ba63e-6ebb-456f-bb53-b39e962fc639 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.615422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.617435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.618920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.619251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.619560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.631909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.633379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.643820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026bfef-16b7-4234-9a4b-89cbe9a72f9e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:31.650463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:31.651497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:31.652539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:31.652539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:31.652539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:31.654765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 251/517] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:31.656306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.685361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.685361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.685361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.685361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.685809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.685809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.762265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc20-69f7-4534-bc27-ac5c1ebb7308 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.784858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 267/518] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:31 2022] POST /identity/v3/users => generated 247 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:31.785816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.809499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:31.814907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.815599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.815888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.816170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.816869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:31.817179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:31.818608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:31.833816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 252/519] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:31.835136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:31.917485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2009-31a9-4c99-98d4-617033741144 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.960110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:31.960701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805831d-ea06-4f36-a157-d0de69947825 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:31.960836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:31.961011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:31.961278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:31.971704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.007435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.008050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.008356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.008614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.010002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.010330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.011745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.030882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 253/520] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.030882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.047634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.049108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.085488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.138607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3902b-79f4-4ca5-b536-5ab50f38a6da None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.156932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 268/521] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:32.158007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.183572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.184362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.184653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.184931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.185658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.185965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.187391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.201471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 254/522] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.202322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.205181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.268021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346f240-147e-4f95-bdb5-60cb5e6d54e1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.307412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.308282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.308862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.309143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.317200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.318697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.320683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfd4da-3a6c-4c44-b71f-ab018a931103 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:32.331212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:32.332192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 269/523] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:32.338600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.339249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.360645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.361284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.361568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.361843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.369700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.371211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:32.386604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:32.387617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:32.388470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:32.389530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 255/524] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:32.390362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.456487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.461233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960779a2-4ccd-4be1-8266-fc652f6e9ce2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.497925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.498745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.499030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.499420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.512839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab401563-2223-431e-85db-1877caf5e9c3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.514012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.515791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:32.533085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:32.534094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:32.539947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:32.539947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:32.539947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:32.539947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 270/525] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:32.539947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.557583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.558154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.558436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.558756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.565913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.567370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:32.580424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:32.581303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:32.582181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 256/526] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:32.583110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.619270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.655907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5753d7-702f-4884-9ab6-945597b7ef3a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.694106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.694866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.695233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.695603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.696996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.697315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.698850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.702745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52199d-aad3-4d74-ab8f-16277dc6b20a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.742310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 271/527] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.742310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.745605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.746428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.746732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.747879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.748718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.749035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.750475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.757056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.778678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 257/528] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.778678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.867585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a0781-7af5-4ffd-9e45-58afde39a508 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.895230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:32.906554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cf4c6-b78a-4955-8fa7-2063ef3338a3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:32.920543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.921585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.921585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.921766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.922405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.922854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.924421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.939315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 272/529] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.940571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:32.948013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:32.948636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:32.948914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:32.949181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:32.949891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:32.950190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:32.951755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:32.971017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 258/530] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:32.971017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.029329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.069132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e502c-2e6f-41ea-8364-cfe4aab2201d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.081442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874d48e-e16e-4b90-84b9-d1fd11537bb2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.106158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.106800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.107081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.107345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.121469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.122001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.122278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.122548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.123306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:33.123622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.125055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.131435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 259/531] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:33.132865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.166825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.207743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.209255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.270082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0a0f7-add6-4faa-825c-354ac0b72121 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.309540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.314007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.314682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.314971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.315248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.324876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.326339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.326882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 273/532] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:33.331375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:33.339457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:33.340352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:33.341608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 260/533] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:33.342060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.432073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.458384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027d6d-f57d-4c3a-b874-5bf3aca3b42a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.466390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9144d20-c8ba-4c41-b916-2df453029534 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.497944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.498564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.498897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.499180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.500102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ee70da9d2f4451ba0d2133653b7d144) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:33.500421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.502649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.506079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.506988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.507465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.507950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.510770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 274/534] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/role_assignments?user.id=4ee70da9d2f4451ba0d2133653b7d144&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:33.511859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.533532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.535424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:33.563938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:33.564884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:33.565789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:33.565789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:33.565789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:33.569410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 261/535] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:33.576559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.591216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.729902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e9fa-7d40-4087-808b-f25d7f574038 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.804311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2905af-fa80-4d75-9e35-a6c352c88cb3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:33.813598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.814546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.814878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.815244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.816049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:33.816352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.817946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8e11-60e4-40ca-88d8-b24b247a61f2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.837562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:33.842006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 275/536] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:33.843942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d167-8ad0-489a-9f01-5834984e8787 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.844485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d167-8ad0-489a-9f01-5834984e8787 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.850199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d167-8ad0-489a-9f01-5834984e8787 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.850199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 276/537] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:33.850398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:33.850398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.850398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.850398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.850700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:33.850700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:33.851574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0e36e-11fd-4ee2-8ba9-919d53ecd7f5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:33.851899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d167-8ad0-489a-9f01-5834984e8787 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.873823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 262/538] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:33.873823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:33.873823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:33.873823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:33.898342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:33.901139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:33.987357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 263/539] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:33.988584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:33.993350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:34.008266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d167-8ad0-489a-9f01-5834984e8787 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.068890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.069536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.069818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.070091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.070869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4710a6cf72564ff7abf8d35fa8e90a5b) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:34.071201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:34.073800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:34.090383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 277/540] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/role_assignments?user.id=4710a6cf72564ff7abf8d35fa8e90a5b&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:34.142671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68653fb-3fce-4d29-8423-5d89f3a436e4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.185639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.186686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.187167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.187473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.188436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:34.203716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:34.205222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:34.220733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:34.221656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:34.222489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 264/541] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:34.229328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:34.312707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:34.362745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15cb54-6656-4f27-95e5-b351e42b4d4c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.410443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.411099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.411382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.411657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.412361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:34.412673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:34.414127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:34.433937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 278/542] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:34.440711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:34.471811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c961961-2ec0-41cb-ada6-c763abab8ba0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.481457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:34.510799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.511435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.511714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.511988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.531367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:34.536060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:34.536931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:34.537710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 265/543] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:34.543677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:34.620906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:34.659194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abac28b-d31f-458b-9aae-421e7eaba27b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.726959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.728412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.728702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.728983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.731036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=ddd1cf9d4721422194e5ec876b819033) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:34.732420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:34.733654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:34.741931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be877fe-5632-482a-ad01-432d99914262 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:34.747119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 279/544] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:34 2022] GET /identity/v3/role_assignments?user.id=ddd1cf9d4721422194e5ec876b819033&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:34.789673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:34.790904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:34.791188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:34.791463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:34.792150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:34.792454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:34.793851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:34.806323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 266/545] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:34.812181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:34.944795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:34.964450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dd3b1-7a67-4475-83a8-4f176177a860 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.004440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:35.005162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.005447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.010755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.017013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:35.018567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:35.034394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:35.035425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:35.037118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:35.037118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:35.037118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:35.037118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 280/546] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:35.042314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:35.185068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:35.188643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f6b8e-8e65-4fce-b596-9a7e44bb6001 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.226037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:35.226674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.227085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.227377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.228840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:35.229149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:35.230854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaeaf8-8f88-4a51-9807-02c26b260fec None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:35.243762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 267/547] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:35.249647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:35.271574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ea3c-a918-44d2-8db4-da460c8001ca None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.271884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ea3c-a918-44d2-8db4-da460c8001ca None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.272130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ea3c-a918-44d2-8db4-da460c8001ca None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.273734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 268/548] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:35.291734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.291734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.291734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.310947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:35.312329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:35.383739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 269/549] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:35.428433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9850852-69c9-4934-8f5a-bfa461f77365 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.478571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:35.483646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:35.484343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.484636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.484910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.500252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:35.502134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:35.520080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:35.521063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:35.522560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 281/550] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:35.527532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:35.647032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:35.651949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9998be-aa5b-4bb0-9453-23f7651f82ce None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.689728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:35.690346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.690637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.690913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.691744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:35.692048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:35.693486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:35.695432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:35.717698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 270/551] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:55:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:35.723935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:35.835183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a83271-44bd-4592-9958-4039a44f5190 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.873642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:35.874262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:35.874542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:35.874968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:35.954838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e56d5-24f2-4240-b7fd-3a6c507ede9b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:35.958292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.033444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.034082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.034360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.034644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.053775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.056403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.075308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:36.080882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:36.081787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:36.083103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 282/552] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:36.090821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.167683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 271/553] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 05:55:36.173256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.231303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.254287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.285471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26487971-26ab-45b8-8be2-2d750715e98b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.325071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.325708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.326000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca80385-16cd-47eb-9ff0-f36da302223e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.326108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.326342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.338186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.339929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-56c5dbe8-0d15-4a57-b515-b60f12859332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:36.361564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:36.362750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:36.363934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:36.363934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:36.363934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:36.365954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 272/554] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:36.367432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7e888-8fc1-43a8-9f3e-da36e280930f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.367868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7e888-8fc1-43a8-9f3e-da36e280930f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.368944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7e888-8fc1-43a8-9f3e-da36e280930f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.374943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 273/555] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:36.376225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7e888-8fc1-43a8-9f3e-da36e280930f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.380427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.381145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.382171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.382171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.382694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:36.383096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.385278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6a440-aafc-4ea4-b34d-08d50cf165fd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.395519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 283/556] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:36.396131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.396563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.396934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.417614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:36.419034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:36.490465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 284/557] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:36.495803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.504932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.509555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7e888-8fc1-43a8-9f3e-da36e280930f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.558448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.559522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.560033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.560509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.560995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.562705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:36.563267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.564934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219420c-0295-40d5-a2a6-5bc11f8c9f52 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.578709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 274/558] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:36.580341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b550-218c-4fb3-baef-13576c6772b4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.580856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b550-218c-4fb3-baef-13576c6772b4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.581329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72b550-218c-4fb3-baef-13576c6772b4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.588014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 275/559] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:36.589344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b550-218c-4fb3-baef-13576c6772b4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.623243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0513b7-1a67-48a4-831b-d5ee4389f1b3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.660101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.660752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.661063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.661281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.672216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.673680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d04b36be-6a68-4426-9e8c-c18a18e46922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:36.690096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:36.691144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:36.692002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:36.692002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:36.692002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:36.694920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 285/560] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:36.696789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.697099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.697600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.708392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72b550-218c-4fb3-baef-13576c6772b4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.717623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:36.718844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:36.751265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3554-2025-4893-866e-272528011969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:36.752307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3554-2025-4893-866e-272528011969 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:36.752789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3554-2025-4893-866e-272528011969 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:36.753417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c3554-2025-4893-866e-272528011969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:36.761959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c3554-2025-4893-866e-272528011969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:36.763982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c3554-2025-4893-866e-272528011969 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-483c3554-2025-4893-866e-272528011969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:36.781791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:36.782792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:36.783681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:36.784105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 276/561] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:36.789265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3554-2025-4893-866e-272528011969 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.816771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:36.836491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 286/562] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:36.837831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:36.998202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c3554-2025-4893-866e-272528011969 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:36.998804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.044112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.044735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.045020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.045296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.045945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.046248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.047656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.083184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 277/563] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.083475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.155694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862ed98-5e50-4e30-a189-3caf060c76cc None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.190240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.190891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.191174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.191451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.193887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.202332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:37.207173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:37.210646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 287/564] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:37.211488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.215053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f090374-e6ff-435f-b483-ab5d52f5fe98 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.250723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.251335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.251765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.252222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.252923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.253228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.254690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.266549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 278/565] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.267167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.310365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.325165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a15a27-dd76-496c-8892-52d3e7d7903f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.359709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.360347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.360630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.360905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.367896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.370079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.371394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e350aee-1e2f-4239-926d-780d8801ace6 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-351b851c-a9d4-4eeb-b3fa-7ec3ea7acc9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:37.381434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:37.382585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:37.384023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 288/566] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:37.384966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd63-f87c-4bae-9a7d-f37348c42fe8 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.384966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd63-f87c-4bae-9a7d-f37348c42fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.384966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd63-f87c-4bae-9a7d-f37348c42fe8 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.385811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 289/567] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:37.386505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdd63-f87c-4bae-9a7d-f37348c42fe8 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.410991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.411833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.411949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.412142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.420452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.424102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:37.428806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:37.449884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 279/568] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:37.450918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.493962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdd63-f87c-4bae-9a7d-f37348c42fe8 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.527433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.528236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.528395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.528580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.529245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.529550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.530966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.533667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853e879-3daf-4297-b705-1133ecbf48f9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.538774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 290/569] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.539570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.553833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.568108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.568709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.568988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.569260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.569983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9d3c0f71e8984988b432390ec99977de) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.570292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.571996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e285d2-11f4-41c7-996d-8aaaf4060f93 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.577325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 280/570] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/role_assignments?user.id=9d3c0f71e8984988b432390ec99977de&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.578458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.578774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.579048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.598423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:37.599684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:37.670374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.674349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3667-7f6c-466a-af1d-419f450b6c2b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.686658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 281/571] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:37.686658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.721695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.722429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.722836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.723175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.724089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.724460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.726879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.754667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 291/572] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.754667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.825231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6139c-005c-411a-9f04-6b29d1b35681 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.861864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.864316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.865164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.865634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.866821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.867781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:37.868576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.870284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.880182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 282/573] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:37.881462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:37.907472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e30e8-88aa-472b-ad44-04d65998bb3d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:37.945864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:37.947671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:37.948161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:37.948658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:37.961475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:37.964155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:37.964283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:37.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:37.983266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:37.984128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:37.984128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:37.984128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:37.984128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 292/574] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:37.984358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.012590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34fbc1-4987-4102-83e3-e1b22c02753a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.053855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.054543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.054846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.055119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.066875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.068366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:38.086429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.087930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:38.089010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:38.089010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.089010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:38.090761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 283/575] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.091611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.102111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8399f-9826-416b-b163-0777b51fdb6e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.123787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.144068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.144712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.144992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.145268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.158496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.159993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:38.174187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.175364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:38.176546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:38.176546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.176546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:38.178962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 293/576] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.180798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.206638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce46c2-d044-4b31-ac35-1baa982d61fa None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.216977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.244380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.245100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.245379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.245651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.256815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:38.262481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.263674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 284/577] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.264874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.300885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cdbc0-2e65-4205-a89d-9180a977c7b9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.335548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.336223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.336459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.337235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.337893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:38.338246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.339673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.357012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 294/578] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:38.357012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.376979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231eca0-9dbf-4672-8e25-5de4d98b8704 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.389026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.411561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.412169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.412486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.412750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.413380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:38.413674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.415645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.427546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 285/579] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:38.428340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.457715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.471478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966320b8-7a77-4399-9912-36c41cec60b8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.507779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.508386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.508704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.509254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.510087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:38.510393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.511811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.523694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 295/580] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:38.525665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.543157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac45dc-b956-48e3-819c-9cec86220fd5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.582498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.582498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.582874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.584110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.584110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:38.584110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.585454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.594172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 286/581] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:38.594862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.640526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.651006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31210b9d-0393-4b42-b514-823cf8550c18 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.693107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.694007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.694295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.694566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.697852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.703460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.704925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.707457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbe40-4512-45c5-bf29-b620c0a4dd96 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:38.716633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.717959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 296/582] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.719374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.748842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.749489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.749774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.750050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.757026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.758477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:38.770828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.771931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:38.773499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 287/583] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.778235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.845166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18eeccc-9c75-44f0-990d-6a9a303ef95e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.884247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.885028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.885316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.885618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.890554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8a1b8-eb45-4b2d-b178-508bbaa2b94c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:38.897318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.899018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:38.912474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.918754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 297/584] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.933410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:38.940722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:38.946131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:38.946131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:38.946131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:38.953343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:38.954908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:38.970221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:38.971463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:38.972584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:38.972584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:38.972584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:38.972584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 288/585] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:38.976772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:38.980172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.034478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d54a5b-b7ac-4550-afd7-64c5a4ef0468 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.072305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.072924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.073210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.073600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.074349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.074679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.076155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.087333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9610a2-4029-4960-8220-aa9775846a8f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.099292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 298/586] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.099292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.123973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.124593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.124878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.125145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.125839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.126149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.127608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.145348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 289/587] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.145348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.173906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.209882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68575297-0775-4f33-9d63-318fb630bf5d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.216422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.246997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.248019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.248583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.249857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.251167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.251646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.253523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.259441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7107033-d000-4543-8291-165fb1f74a51 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.273657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 299/588] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.275770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.300690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.301297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.301570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.301842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.302480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.302810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.304176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.316078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 290/589] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.316861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.391931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f932f-142e-4799-af33-1f4395e6303d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.420430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.424837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f169ec-6d73-43cd-8b42-c9ae3ebb6faa None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.429695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.432218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.432218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.432218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.443264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.444838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.449568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.461195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:39.462090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:39.463336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:39.464363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:39.464363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:39.464363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:39.464534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.464534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.464534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.464732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 300/590] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:39.464981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.485105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=9dd40d8b3cb543feab8424f1d87b7126, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.486571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.547116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 291/591] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:39 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:39.548811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.574268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a3cca-35d2-4a5a-898f-39b75d321886 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.615184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.615849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.616142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.616429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.629037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.630652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:39.646923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:39.647947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:39.648811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:39.648811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:39.648811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:39.648811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 301/592] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:39.649069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.678816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef972b0-d3ec-4c18-b972-2fcb9d81cb8e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.688107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.726027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.726882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.727309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.727685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.735950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.737540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ce1fa57e-8bb5-4be9-9ccb-3fbf1c6c3fda None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:39.751121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:39.752282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:39.753433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:39.754832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 292/593] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:39.756086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723637-bd74-4f6d-b011-4a24dd0f7d14 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.759273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723637-bd74-4f6d-b011-4a24dd0f7d14 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.760067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723637-bd74-4f6d-b011-4a24dd0f7d14 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.762352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 293/594] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:39.763667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a723637-bd74-4f6d-b011-4a24dd0f7d14 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.768266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeca10f-7f8d-4cf7-b24c-8cddefa6d77d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.804125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.807909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51939543-4348-4427-b106-94875d094fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.808532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939543-4348-4427-b106-94875d094fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.808817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939543-4348-4427-b106-94875d094fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.809101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939543-4348-4427-b106-94875d094fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.809756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51939543-4348-4427-b106-94875d094fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.810065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51939543-4348-4427-b106-94875d094fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.811485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51939543-4348-4427-b106-94875d094fe9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.823950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 302/595] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.826055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51939543-4348-4427-b106-94875d094fe9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:39.885164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a723637-bd74-4f6d-b011-4a24dd0f7d14 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.924908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.925517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.925799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.926073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.926760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.927062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.928459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c34d6-2b1e-4281-8896-994924f2231a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:39.931772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:39.946696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 294/596] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:39.946910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51939543-4348-4427-b106-94875d094fe9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:39.947022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.947022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.947022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.961937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:39.963442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:39.987895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:39.989416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:39.989965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:39.990395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:39.991449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:39.991935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:39.994408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.013572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 303/597] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 05:55:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:40.014907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.041044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 295/598] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:40.042071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.066711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.147935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ffed-166a-46ee-9e27-c95e10b036b4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.157742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3b3d7-5b82-4372-b42f-5c8ef62c7e66 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.187305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.188201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.188599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.189074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.198035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.199790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.202996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.203882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.204316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.204734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:40.212629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:40.214262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:40.215452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:40.221057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 296/599] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:40.221057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.221696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.224246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:40.246497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:40.247605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:40.248660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:40.255645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 304/600] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:40.256856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.269543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.314382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.342054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38b160-596b-4b36-b6e4-edb6194dd122 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.381247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4dc91-f9a4-45cb-a482-cccaec4383ed None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.386932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.387677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.388031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.388381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.400351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.402142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.417532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.418149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.418432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.418736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.419440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:40.419748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:40.420179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:40.421262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:40.422388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:40.422388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:40.422388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:40.422596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.427360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 297/601] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:40.427588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.439865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 305/602] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:40.441409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.514975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.541559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79476de7-5090-4210-a6f7-f5a3390bc46b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.551095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad452cd2-0d27-4ee9-96f5-a4e07704093a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.578030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.578727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.578955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.579232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.579970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:40.580255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.581699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.583714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.590274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.591123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.591452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.591831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.600960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 298/603] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:40.600960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.603590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:40.608937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:40.610194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:40.611352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:40.612847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 306/604] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:40.612847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.720522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac2e2-9afb-48fe-a7e8-0d0df03e764c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.720877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561b3b1-3006-47ff-89fe-7db7847bafe2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.750210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.759167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.763848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.763973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.763973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.764146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.764232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.764314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.764314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4710a6cf72564ff7abf8d35fa8e90a5b/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.764479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:40.764479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.764479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.775553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 299/605] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:40.776551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.783944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=4710a6cf72564ff7abf8d35fa8e90a5b, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.785419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:40.833377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:40.869543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 307/606] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:40 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4710a6cf72564ff7abf8d35fa8e90a5b/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:40.870323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:40.880184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c766-eb75-47c9-aa80-48a05f77eeb0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:40.926493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:40.927368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:40.927637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:40.927912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ee70da9d2f4451ba0d2133653b7d144/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:40.949375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=4ee70da9d2f4451ba0d2133653b7d144, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:40.951284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.009928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fa5c0-9692-47ae-b3e5-490532f1bd5a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.050440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.051097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.051391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.051673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.052353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:41.052663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.054098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.055163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 300/607] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:40 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ee70da9d2f4451ba0d2133653b7d144/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:41.057703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.065816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 308/608] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:41.072104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.126583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:41.194233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2345ad-7f06-4689-8e94-570927c9872c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.210410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a1200-2e1a-41c0-8196-113cd52b11e5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.214842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:41.242837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.244321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.245799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.246237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.278501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.280047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.296242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.296891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.297171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.297443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.306997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:41.307820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:41.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:41.309817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:41.309817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:41.309817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:41.309983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.310059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 309/609] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:41.316525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:41.327555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:41.328680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:41.329602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 301/610] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:41.334963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.462954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:41.485678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fec97-a80d-4011-b098-0395d7bfb9a6 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.487709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef5adf-0680-4eac-998c-373d8bec1bb5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.525584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.526197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.526481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.526787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.527496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:41.527796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.529252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.543261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 302/611] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:41.551846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.576766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.577430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.577807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.578096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.578791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:41.579106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.581355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.607080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 310/612] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:41.611078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:41.613934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.738803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51771578-d7d5-41e1-b936-ee28535dd426 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.780432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ae70-02a4-4fae-9544-1a74b7ab8299 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:41.781096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.781768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.782050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.782326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/ddd1cf9d4721422194e5ec876b819033/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.798581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=ddd1cf9d4721422194e5ec876b819033, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.801612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512fc4-b40b-4eb4-9504-316bde49caa9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.824597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:41.825472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.826015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.826432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.834088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:41.835909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-80aa723c-ed77-4a16-b7d0-0ce84a23410e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:41.849942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:41.851130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:41.852328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:41.859136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 311/613] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:41.860019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289364-1b0c-4a8d-8e62-514664379b5c None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.860361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289364-1b0c-4a8d-8e62-514664379b5c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.860669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc289364-1b0c-4a8d-8e62-514664379b5c None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.862153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 312/614] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:41.863069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289364-1b0c-4a8d-8e62-514664379b5c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:41.878573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:41.903215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 303/615] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:41 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/ddd1cf9d4721422194e5ec876b819033/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:41.911605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:41.911971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:41.912270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:41.953172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:41.956010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:42.084074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc289364-1b0c-4a8d-8e62-514664379b5c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.087637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 304/616] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:42.128139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.129152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.129640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.130180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.131271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:42.131814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:42.134098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:42.154526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 313/617] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:55:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:42.159973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.170897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:42.276529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f87e5-fab5-4810-b5b6-05d3f5eee837 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.326151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.326831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.327117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.327395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.339812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:42.341192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:42.366745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:42.368492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:42.370142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:42.370142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:42.370142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:42.370142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 305/618] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:42.374417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.427128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.471194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:42.508448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7470e8-239c-4021-933d-8ae9585a47c3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.547178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.548047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.548620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:42.549431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.549784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.550540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:42.550874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:42.552275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:42.573533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 314/619] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:42.579099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.613009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203394-d92d-4d37-8cd4-88d8ce6b86f3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.678809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.679449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.679736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.680010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.697741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:42.703641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:42.704781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:42.705754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 306/620] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:42.715620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.744228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9824-4d19-4eda-b36b-aaf389e3fc13 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.781931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.782572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.782886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.783168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.786957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:42.791943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:42.793720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:42.806834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:42.809285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:42.810353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 315/621] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:42.815779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:42.944283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7775d0-aafa-47e4-a8f1-90caf0fef52e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.949849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0befdc-59f3-4043-9c23-6c6fa27ff864 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:42.977808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:42.988530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:42.989302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:42.989684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:42.990099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:42.990937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:42.991306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:42.993482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.017659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 316/622] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:43.019744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.020504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.020915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.021278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.022043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:43.022444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.024009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.025601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.047269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 307/623] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:43.052812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.123008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:43.167661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51836b2-01b6-42a7-b633-096f0dc7542b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:43.208708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.209569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.210037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.210405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9d3c0f71e8984988b432390ec99977de/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.225492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ec5e8-199a-4640-b5b6-c8fbadef193f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:43.228596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=9d3c0f71e8984988b432390ec99977de, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.230182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518848f3-47cf-4ac8-aba8-af5c2f2ebc9f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.262436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.263094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.263535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.263753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.276775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.278826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-27b19848-9108-4ecd-9216-1d020eaab0d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:43.293490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:43.294452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:43.295701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:43.295701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:43.295701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:43.295701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 308/624] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:43.296270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343ef3-9a9d-4db2-8247-8e3d39f4e8e1 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.299371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343ef3-9a9d-4db2-8247-8e3d39f4e8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.299371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343ef3-9a9d-4db2-8247-8e3d39f4e8e1 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.299371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 309/625] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:43.303512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343ef3-9a9d-4db2-8247-8e3d39f4e8e1 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.338261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:43.375025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 317/626] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:43 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9d3c0f71e8984988b432390ec99977de/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:43.383059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.384238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.384752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.409796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:43.411531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:43.448473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343ef3-9a9d-4db2-8247-8e3d39f4e8e1 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:43.489797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.490461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.490777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.491105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.491819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:43.492120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.494130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d8b6-16fa-4fe6-8c27-954dfe65bc7c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.512367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 310/628] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:43.512367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d619-c8f1-4596-b3ae-3aebf79f809e None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.512367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d619-c8f1-4596-b3ae-3aebf79f809e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.512367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837d619-c8f1-4596-b3ae-3aebf79f809e None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.513186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 318/629] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:43.516857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 311/629] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:43.519234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.519519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.519796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.520679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.551816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:43.553010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:43.570151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:43.658708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 312/630] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:43.675102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36a0b4-a03b-48ef-8a8f-9492eaeec09f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:43.730442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.731039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.731330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.731613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.740733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.742204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:43.759196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:43.760374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:43.761900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 319/631] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:55:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:43.765874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.800284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:43.805041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:43.901296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03b835-a9c9-4b85-ae15-0a56adb281da None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:43.955342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:43.956268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:43.956694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:43.957114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:43.958000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:43.958478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:43.960182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:43.966975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:43.978412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e9e07-959c-4855-8fdc-ff88451c75da None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.001150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 313/632] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:55:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:44.006559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.019107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.019753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.020035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.020829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.040333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:44.045968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:44.046914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:44.048205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 320/633] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:44.048943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.119007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.138185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c48c2-82a7-49ef-a775-93554182552c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.165997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07154392-2a1e-4781-9855-3a9452836d7c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.175304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.176149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.176627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.177085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.203848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.204763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.205123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.205508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.217398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:44.222785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:44.223719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 321/634] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:44.230097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.245010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.260907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.262210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.345509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.347435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014f20c-2948-4c86-9041-34d2621c8ea9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.368393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 314/635] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:44.369226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.388978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.389991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.390442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.390894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.391563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:44.391870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.393239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb5c7d-55af-45c0-9877-090bde4e5a4d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 322/636] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f448-f717-458f-afad-3fcd73ebdc38 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f448-f717-458f-afad-3fcd73ebdc38 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f448-f717-458f-afad-3fcd73ebdc38 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 323/637] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:44.406478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f448-f717-458f-afad-3fcd73ebdc38 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.477323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c914-fa91-4518-beec-f52fa643fe4c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.481564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.512446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.512981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.513260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.513532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.514196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:44.514497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.515895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.516585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f448-f717-458f-afad-3fcd73ebdc38 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.524081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 315/638] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:44.525558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.557599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.558237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.558520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.558823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.569667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.571737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.573792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-47b97bad-96a4-47bd-9909-26f25b0c3afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:44.586101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:44.593191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 324/639] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.594146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.610130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:44.611611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:44.645064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f937-a6b3-4659-a4e2-27592d16c1bc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.685984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.686714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.686916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.687189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.688011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 325/640] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:44.689107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.709979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.712627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:44.732167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:44.733283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:44.734161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:44.734161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.734161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:44.738760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 316/641] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:44.741650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.757263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.807498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99394533-ab21-4108-8987-1402dc9b1bf2 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.856583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.857784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.858226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.858930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.859986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:44.870241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b9aa5-7437-471e-813d-894345430b6f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:44.875808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.876223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:44.902807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:44.903823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:44.904726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:44.904726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:44.904726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:44.904726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 326/642] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:44.904726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:44.922164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:44.922836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:44.923112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:44.923385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:44.924035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:44.924333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:44.926109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:44.946710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 317/643] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:44.947981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.034186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd94a-7c20-4a6e-83f6-c64d931c5d47 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.043744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.063776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97a46c-7401-4244-bb00-0eb7eb904836 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.070559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.072015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.072289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.072556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.073212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.073520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.075023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.089110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 327/644] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.090258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.105578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.106189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.106465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.106769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.107416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.107713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.109090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.122357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 318/645] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.123252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.141738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.204164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722c8a9-65f0-4c80-be43-2d51da4c79dd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.230830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1eab7-eab3-411f-8fe0-69e46530e944 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.239265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.239907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.240195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.240510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.247965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.249421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:45.261282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:45.262184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:45.263274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 328/646] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:45.263793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.271052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.271690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.271969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.272240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.283438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:45.288645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:45.289964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:45.291164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 319/647] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:45.292373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.297350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.373875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.375514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7caf39-dfcf-4b96-abfd-728399bcaffb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.403043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f227f6-c607-4da6-a5ec-f337bfc4025e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.412882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.414118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.414118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.414118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.422485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.424003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:45.437090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:45.438084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:55:45.444040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:45.444584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.444584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.444584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.444584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.444835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 329/648] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:45.444835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.453641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.455143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:45.466460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:45.467757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 05:55:45.468926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:45.472250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 320/649] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:45.473110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.541538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.557665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ef086-2dee-4fb0-a05c-2c11f930eb7e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.576906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d621139-754a-4cbf-a1dd-17c6bc35d800 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.591772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.592441 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.592720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.592994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.593701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.594003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.595479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.607302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.607444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 330/650] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.608325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.613695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.614310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.614596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.614899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.615569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.615875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.635863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.635863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 321/651] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.635863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.708464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ce7a-93f3-4e41-870f-8d5241df9df3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.728511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1018a4-f53c-4057-a4cd-d555e6a36876 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.744083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.744705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.744996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.745274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.746021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.746320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.747792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a5213-f762-4233-bc15-9c1c4be3c9f4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.762516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.763148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.763423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.763692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.764410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.764712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.765851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 331/652] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.766125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.767115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461781-0640-400a-a878-4e59332f0703 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.767407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461781-0640-400a-a878-4e59332f0703 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.767688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461781-0640-400a-a878-4e59332f0703 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.770746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 332/653] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:45.770746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461781-0640-400a-a878-4e59332f0703 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.774134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.786227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 322/654] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.791864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.842386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:45.876445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461781-0640-400a-a878-4e59332f0703 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.895603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba306-231a-4110-94fa-07d526b9d94d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:45.918196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.918851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.919139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.919418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.920160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:45.920475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.921929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.928157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 333/655] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:45.928944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:45.930974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:45.931600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.931880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.932152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.951079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:45.952895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-dbd901a0-1ec5-42a4-9e3a-9ee675ed8eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:45.969531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:45.971075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 323/656] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:45.972644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:45.973407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:45.995123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:45.996360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:46.013547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.052000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d33250-2186-4d6e-9f6d-67b9d9f08e5c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.087023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 324/657] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:46.088812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.099450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.100144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.100436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.101084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.129275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.187760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.189725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.227855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a5c6-0659-4f7c-9c0d-f6779a828e69 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.269167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.270067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.270356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.270650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.285616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.325319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 334/658] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:46.326632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.361987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.363480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.401654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.436409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50793ab-5212-4968-a73e-412c0459aaf8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.475462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.475993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 325/659] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:46.476724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.481637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.481934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.482218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.483235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:46.483450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.484881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.502610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 335/660] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:46.502610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.531009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.597053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc422-5a56-4bbc-9bc0-b984fb7d49ee None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.607191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0e25b-641c-48fb-afa4-d124226b8a8a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.636549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.637187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.637463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.637735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.639776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.642052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.642719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.643006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.643283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.650644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.652304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.654880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:46.660058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:46.661029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:46.661860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:46.666115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 336/661] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:46.667056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:46.670886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:46.671957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:46.672860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:46.672860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:46.672860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:46.677131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 326/662] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:46.677934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.772299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.786035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc53e4-4b9d-4599-98f9-e0e0f38985d0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.790347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037b483-b265-4f60-b223-793543d58ac3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.823203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.823945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.824235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.824519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.829453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:46.830362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:46.830896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:46.831501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:46.835066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.836586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.840573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:46.842179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:46.851982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:46.852999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:46.854983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:46.854983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:46.854983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:46.854983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 337/663] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:46.854983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:46.858801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:46.859712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 05:55:46.861657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:46.864591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 327/664] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:46.865565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:46.919614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:46.980557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9f750-08f2-4c49-ab8f-a260556ecf21 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:46.982951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c4fd5-944d-4ca8-ad47-0a0ca0c1078e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.017130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.017832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.018113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.018406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.019246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.019554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.019664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.020258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.020545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.020824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.020928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.021492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.021801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.023530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.037568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 338/665] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.037568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.040855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 328/666] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.044247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.050977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.152181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aac7cb-cb61-4e0f-a36d-67019769edac None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.152436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187016e-b62f-4d57-9723-e3bcae558917 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.161791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.190779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.191121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.191390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.191671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.191781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.191984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.192094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.192267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.192766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.192986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.193095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.193299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.194478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.194696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.215416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 339/667] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.215416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.220099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 329/668] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.225335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.285074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.327634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a543f-5e86-4add-964f-4c1ff30abecd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.339279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f214a-1fcc-4a6b-9b6c-c31df260e5ed None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.370585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.371259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.371544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.371823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.382293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.382466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.383727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.384114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.384472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.384586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.391663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.393350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:47.401118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:47.402319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:47.403363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:47.403363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:47.403363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:47.406921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 340/669] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:47.407628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:47.408518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:47.409408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:47.409825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.413805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 330/670] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:47.415380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.447954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.527350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bccbf-e66b-4360-91fc-278740d0726a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.536626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b89edd-5ea4-44be-88dd-72b0266bd5dd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.543611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.567515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.568303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.568593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.568876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.578043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.579137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.579137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.579137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.582115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.584605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:47.598109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:47.599181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:47.600062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 341/671] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:47.600543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.672913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.675249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.710505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.724826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d17865-5946-4811-a1d4-b7d5e185d138 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.770839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.771600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.771892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.772170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.773204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.773832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.775730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.802435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 342/673] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.802634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 331/673] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:47.803078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.803182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.846348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.910720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b97db7-0111-4356-a705-1125553ae63e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.925240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd38ac-156c-4c5b-bede-2f119e447177 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:47.935702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:47.947677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.948285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.948571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.948841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.949550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.949876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.951326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:47.963694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 332/674] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:47.964526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:47.980339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:47.981447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:47.982309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:47.982850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:47.983997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:47.984713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:47.986722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.004688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 343/675] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:48.005790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.073429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57eca2-4510-43ec-a7f2-8c4e3a5778ee None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.109404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.110311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.110589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.110883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.112888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.119509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.121018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce05de-743d-465f-8233-71a55f947e5a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.121122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:48.132066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:48.133248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:48.134724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 333/676] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:48.135338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.160010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.160653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.160940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.161221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.172188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.173644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.181567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:48.188471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:48.195039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:48.196830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:48.196830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:48.196830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:48.196830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 344/677] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:48.196830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.236331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b881824-ffad-4380-8c7d-6fac588dde9c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.271130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.271733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.272016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.272288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.273012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ee70da9d2f4451ba0d2133653b7d144) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:48.273318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.275442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355cca1-735a-4f97-a325-1745d680556f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.281074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 334/678] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/role_assignments?user.id=4ee70da9d2f4451ba0d2133653b7d144&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:48.282251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38671-734a-4a18-b4b6-d89e375b9f29 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.282538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38671-734a-4a18-b4b6-d89e375b9f29 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.282857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af38671-734a-4a18-b4b6-d89e375b9f29 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.284028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 335/679] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:48.284728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af38671-734a-4a18-b4b6-d89e375b9f29 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.304424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daafde0-2aaf-494c-9068-243665b5781a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.348197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.348709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.348995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.349275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.350038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4710a6cf72564ff7abf8d35fa8e90a5b) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:48.350350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.351967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.370656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 345/680] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/role_assignments?user.id=4710a6cf72564ff7abf8d35fa8e90a5b&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:48.370656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.384663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.439987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af38671-734a-4a18-b4b6-d89e375b9f29 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.451216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.492524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.494250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.494786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.495165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.496184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:48.496435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.497982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992324-2909-4ccb-85c8-f3831c085a58 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.517523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b02e4e-9dc2-47de-9860-0c91449d6e6b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.522921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 336/681] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:48.527335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.527335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.527335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.545180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:48.546427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:48.568739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.569360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.569701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.569984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.571125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:48.571509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.572986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.586548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 346/682] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:48.587384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.668863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 337/683] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:48.671862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.685481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.742514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f484e-aa4e-4768-b8ce-c07fb595a7c2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.791068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:48.797838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7d16b-3c13-4300-9fe5-c6d255d26fe7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:48.819193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.820189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.821259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.821779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.868552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:48.870029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:48.870715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:48.871219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:48.880749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.882563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:48.902384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:48.903348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:48.904232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 338/684] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:48.907093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:48.948750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:48.950340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.066047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.084794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81377607-818b-4a5f-af4a-da135205cc74 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.106275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 347/685] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:49.107311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.122899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.123516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.123797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.124163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.124866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:49.125170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.126644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.139407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 339/686] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:49.140358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.159685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.234318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981996b-9b72-42bb-825e-e4bc68ce7b4d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.248716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816a35a-f422-46e5-8218-6cb065c5720f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.275436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.276446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.276885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.277314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.284035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.284664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.284941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.285214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.290120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:49.295331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:49.296474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:49.297469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:49.298136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.298136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.304468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 348/687] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:49.305603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:49.313017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:49.314153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:49.315297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:49.315297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:49.315297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:49.315297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 340/688] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:49.315297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.343458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.384808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.420102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f8c1d-7783-424b-ab0d-60a7cdc43f2d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.425217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38322c-0f25-4d73-86a3-fd3e225892cd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.457034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.457643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.457921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.458191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.458884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:49.459184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.460554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.467414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.468314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.468833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.469315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.470324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=ddd1cf9d4721422194e5ec876b819033) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:49.470896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.472664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.475008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 341/689] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:49.475927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.486916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 349/690] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/role_assignments?user.id=ddd1cf9d4721422194e5ec876b819033&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:49.488323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.606057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980d8d1-b3fa-4479-a8c7-d9f301418a28 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.634252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.642543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a44b8-84ba-4d64-9b94-a967e4c1c02b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.661455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:49.669870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.670501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.670813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.671087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.671739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:49.672039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.673411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.687328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 342/691] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:49.692653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.693210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.694044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.694656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.695167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.712213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.714724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:49.732987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:49.734103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:49.735528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:49.735528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:49.735528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:49.737623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 350/692] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:49.744212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.828293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccc8ab-aecb-41fa-95ab-4198ecb61fe3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.874453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.875239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.875408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.875669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.884230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.885685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.904450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638c3d1-3b3b-41f7-815f-5f350f3a3639 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:49.909088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:49.909991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:49.910921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:49.913352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 343/693] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:49.920812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.950803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:49.951419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:49.951706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:49.951986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:49.952716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:49.953025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:49.954415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:49.968987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 351/694] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:49.976025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:49.993347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:50.009698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:50.125643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5b61-9124-43d7-89d9-9b0630e13812 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:50.125845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cfc35-6edd-4331-850f-f342d52b9dbb None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:50.184394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:50.185508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.187422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.188551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.198281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:50.200037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.200750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.201521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.202404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:50.202938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:50.204577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:50.207806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:50.209375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:50.223571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 344/695] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:50.225699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:50.244406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:50.245405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:50.246346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 352/696] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:55:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:50.255644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:50.344971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb06ea-0fbc-4a4b-81a3-cc62c49dd30a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:50.363064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:50.382064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:50.382980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.383976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.384406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.385274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9d3c0f71e8984988b432390ec99977de) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:50.385724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:50.387480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7383-9003-40cf-bb10-035ba6a7ae3e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:50.394209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 345/697] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:50 2022] GET /identity/v3/role_assignments?user.id=9d3c0f71e8984988b432390ec99977de&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:50.467267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:50.521695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b911fc-2f61-489f-9ffe-b1e29bfdbc35 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:50.568164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:50.569061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.569502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.569920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.571453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:50.571980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:50.573573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:50.583216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00d6e-1497-44c5-a352-418a6fe28fc9 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.583836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00d6e-1497-44c5-a352-418a6fe28fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.584278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b00d6e-1497-44c5-a352-418a6fe28fc9 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.587683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 346/698] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:50.599474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 353/699] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:55:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:50.601337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f142a-a584-4c74-8a9d-bd862b1b92c6 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.601631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f142a-a584-4c74-8a9d-bd862b1b92c6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.601974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f142a-a584-4c74-8a9d-bd862b1b92c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.605680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:50.632936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9b7f142a-a584-4c74-8a9d-bd862b1b92c6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:50.635111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9b7f142a-a584-4c74-8a9d-bd862b1b92c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:50.721548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 347/700] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:50.729970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b688d6-64c8-4ef8-80f1-ecf7f8e1b5cf None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.730261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b688d6-64c8-4ef8-80f1-ecf7f8e1b5cf None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.730540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b688d6-64c8-4ef8-80f1-ecf7f8e1b5cf None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.731812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e3873-5365-4057-9614-72815c7f48a3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:50.732196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 348/701] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:55:50.737124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.737412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.737692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.759539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:50.761292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:50.767423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:50.777988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:50.778921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:50.779210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:50.779487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:50.858448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 349/702] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 05:55:50.877567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:50.879237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.027483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 354/703] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:51.033214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.038328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.085059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.087950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.188700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1418-f9f7-4dbe-b62d-40558683ce7e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.220085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1705f-8163-40cc-9e21-dd93a2bb2f15 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.232841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.234042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.234604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.235229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.246920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.249149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.261066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.261716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.262399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.262719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:51.269627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:51.270819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:51.271922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:51.271922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:51.271922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:51.275154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 355/704] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:51.276370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.282309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.283915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:51.301286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:51.302283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:51.303256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:51.303256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:51.303256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:51.305439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 350/705] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:51.306652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.379063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.404029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5ddef-ec50-4182-a6b8-317625f12014 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.412123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.430602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5eb54-3de9-43f8-98c2-f80e27898d53 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.452276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.452920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.453640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.453933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.454537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.456387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.468917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.469911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.470437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.470938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:51.471046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:51.472002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:51.472836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:51.474805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:51.475324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.477303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 356/706] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:51.477865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.478236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.491879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 351/707] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:51.493233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.610679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea1cce-f017-41b4-925c-fbb3bbfa353d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.615901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e80534-6f37-4712-95c0-1000682704e9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.649647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.650944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.651458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.651956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.652809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:51.653323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.654936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.659969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 352/708] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:51.661003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.670985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.671958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.672460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.672951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.673850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:51.674372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.675726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.676274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.687037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.691210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 357/709] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:51.692337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.796911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669bc1f-eafc-4406-baec-9ba7bfb44fc6 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.798581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9a7d2-2d63-4231-a5f5-183fdb5e9c17 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.836649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.837677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.838198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.838697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.839532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:51.840023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.841630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.842013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:51.842783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.843140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.843476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.844204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:51.845902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf24fb1-f901-4f08-a534-0f2c90e5225b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:51.857036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 358/710] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:51.858356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:51.940969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:51.977061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 353/711] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:55:51 2022] POST /identity/v3/users => generated 254 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:51.992855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5f9a6-93f1-4c59-ab4e-e975a8396be7 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:51.993487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5f9a6-93f1-4c59-ab4e-e975a8396be7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:51.993971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5f9a6-93f1-4c59-ab4e-e975a8396be7 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:51.995402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab9104-14d4-42ee-a07f-33868568f46b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:51.996369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 354/712] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:52.004464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397018-a06e-4270-9653-4481f35fef6d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.005092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397018-a06e-4270-9653-4481f35fef6d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.005569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397018-a06e-4270-9653-4481f35fef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.039039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:52.039848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.040136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.040420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.042488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-91397018-a06e-4270-9653-4481f35fef6d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:52.049386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-91397018-a06e-4270-9653-4481f35fef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:52.049627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:52.050444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:52.064435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:52.065401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:52.066385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 359/713] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:52.072814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:52.142917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:52.159788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 355/714] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:52.208220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e246708-9831-4e5f-9721-acbb09553ff6 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:52.243816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:52.254816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:52.255525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.255812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.256094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.256805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:52.257119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:52.258726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:52.271763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 360/715] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:52.277689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91397018-a06e-4270-9653-4481f35fef6d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:52.390946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91397018-a06e-4270-9653-4481f35fef6d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:52.428718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:52.429261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.429545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.430415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.431207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:52.431519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:52.433025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:52.439489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 356/716] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:52 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:52.453912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:52.591604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a2505-78b9-4b7c-99eb-d54eeb97d8f9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:52.640401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:52.641065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.641354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.641635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] PATH_INFO: `/v3/services/network` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.642228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:52.643640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:52.654761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:52.655903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:52.656972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 361/717] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:52 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:52.663868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:52.755795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:52.811001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04dece-bf02-4627-b563-8da1cae89a0a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:52.847158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:52.847821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:52.848111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:52.848387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:52.849025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:52.849329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:52.850799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dddf95-9af3-460b-8e72-052fad5399d1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:52.856442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 357/718] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:55:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:52.862095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:52.966908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:53.042361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bc1c5-a6fd-483d-bb5e-a9f0c05768ca None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:53.091988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:53.092626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:53.092919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:53.093194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:53.093844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] RBAC: Adding query filter params (type=network) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:53.094152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:53.095572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c42ba-9b7b-41de-ac5d-878aa721492d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:53.101038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 362/719] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:55:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:53.257698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c412b-a355-4037-8c33-2ee12325f207 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:53.258257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c412b-a355-4037-8c33-2ee12325f207 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:53.258603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c412b-a355-4037-8c33-2ee12325f207 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:53.260480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 358/720] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:53.265859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:53.266156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:53.266440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:53.294596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:53.295779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:53.372068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 363/721] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:53.673998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c412b-a355-4037-8c33-2ee12325f207 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:53.809364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c412b-a355-4037-8c33-2ee12325f207 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:53.837567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:53.849988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:53.850641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:53.850934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:53.851423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:53.863102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:53.867921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:53.868947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:53.870878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 359/722] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:53.877577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:53.946024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:54.017655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dae49e-ffba-41a2-b11b-7b7f93b79030 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:54.073149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050574a3-3356-405e-9aa4-583574741288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:54.074049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050574a3-3356-405e-9aa4-583574741288 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.074476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050574a3-3356-405e-9aa4-583574741288 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.074913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050574a3-3356-405e-9aa4-583574741288 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.075728 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050574a3-3356-405e-9aa4-583574741288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:54.076176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050574a3-3356-405e-9aa4-583574741288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:54.077715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050574a3-3356-405e-9aa4-583574741288 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:54.097705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 364/723] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:54.103541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:54.270417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba7b85-a8f9-4822-a065-4b70f31f6263 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:54.321965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ed6a7-0b8f-42ed-9944-c6991d61f129 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.322697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ed6a7-0b8f-42ed-9944-c6991d61f129 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.323245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ed6a7-0b8f-42ed-9944-c6991d61f129 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.324290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:54.324940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.325220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.325490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.327459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 365/724] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:54.333185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.333551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.333807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.342071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:54.343568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:54.355312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:54.356539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:54.366115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:54.367330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:54.368478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:54.368478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:54.368478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:54.368478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 360/725] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:54.375376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:54.453976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 366/726] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:54.475022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:54.581338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b6824-1b90-4b9d-9418-fc3f7ba76f65 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:54.619960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:54.620628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.620913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.621187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.621838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:54.622139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:54.623555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:54.637589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 361/727] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:54.642681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:54.706702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:54.778508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:54.789799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95af87-9bd8-43a0-adc4-06e574ac23c3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:54.831872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:54.832444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.832729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.833005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.853303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:54.855006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:54.874719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:54.875850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:54.877329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 367/728] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:54.892335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:54.924669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:54.932493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7316965-a98d-4343-8dd6-b79fbf465650 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:54.986685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:54.987929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:54.988376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:54.988815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:54.990221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:54.993499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636310a-8a70-439d-86aa-da4b967296e9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:55.046212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.055982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50fb8-3272-457e-ab71-9b72ee625d1a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:55.093548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:55.094094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.094377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.094998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.095449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:55.095754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:55.102404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe4f0f-2e16-4f2b-9a89-e469dfa40a40 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:55.102588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 362/729] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:55:54 2022] POST /identity/v3/services => generated 251 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:55:55.102588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12636b91-e421-4e6c-9bf4-b16bde0efa66 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.102588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12636b91-e421-4e6c-9bf4-b16bde0efa66 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.102588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12636b91-e421-4e6c-9bf4-b16bde0efa66 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.104170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 363/730] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:55.105334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859bbf3-afb4-4844-b22a-6ba47fbe3745 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.105690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859bbf3-afb4-4844-b22a-6ba47fbe3745 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.106029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859bbf3-afb4-4844-b22a-6ba47fbe3745 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.113260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 364/731] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:55.115108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.115462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.115830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.140983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 368/732] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:55:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:55:55.144247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.145255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.146179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.151113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:55.152389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:55.174573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:55.176283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:55.254777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 365/733] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:55.255762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:55.289513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 369/734] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:55.371094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.390868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c9e14-274e-4bfa-9a75-faf972cb910b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:55.432926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:55.433572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.433855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.434131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.511127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:55.512801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:55.545261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:55.635431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.656510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 366/735] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 05:55:55.657547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:55.691395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.717246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c1ab0-cee9-4151-b98a-87777f29520f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:55.756296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:55.757376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.758567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.759322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.771056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:55.777048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:55.778000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:55:55.778970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:55.781241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 370/736] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:55.782206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:55.788711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c89665-432f-420e-b8bd-b499428e2669 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:55.834429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:55.835376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.835838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.836332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.847931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-27c239c7-74bc-41a3-8d9c-38c2283f5f33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:55:55.854081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:55.855712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 367/737] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:55.857061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504151-920b-48eb-8371-d49e46799057 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.858106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504151-920b-48eb-8371-d49e46799057 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.858106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c504151-920b-48eb-8371-d49e46799057 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.858932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 368/738] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:55.859516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c504151-920b-48eb-8371-d49e46799057 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:55.926017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c4076-c2ee-4356-ae11-858d3a1e58b4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:55.938994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.965567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:55.966258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.966483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.966788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:55.978670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:55.980124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:55.982022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-527a0114-dfc6-42d0-9d36-d1c73fbef34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:55.996090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:55.997064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:55.998383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:55.998383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:55.998383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:55.998383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 371/739] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:55.998998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:55.999233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:55.999508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.020388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:56.021585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:56.054143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c504151-920b-48eb-8371-d49e46799057 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.096692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.097326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.097690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.097868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.098512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:56.098842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.100218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.114259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 369/740] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:56.114259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.131010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 372/741] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 05:55:56.131690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.239959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348bf97-4e3f-4605-827b-2c1e4562fea1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.243313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:56.251787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa2690-2862-4846-a235-07a9374bf476 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.280193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:56.282439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.283343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.283823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.284289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.285207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:56.285731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.287339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.291931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.292549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.292910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.293188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.293842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:56.294147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.295859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.299409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 370/742] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:56.300606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.312934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 373/743] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:56.313857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.416070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bc533-f043-43e0-8061-27c71af4ccda None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.453758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.454454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.454770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.455039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.467592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.469044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.473199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e570b-91e6-43a0-a860-22f69b8c55d4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:56.486081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:56.487169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:56.488068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:56.488068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:56.488068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:56.488068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 371/744] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:56.488068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.514723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:56.519455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.520249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.520711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.520864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.534922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.536550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:56.551404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:56.552507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:56.559255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:56.559255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:56.559255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:56.559255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 374/745] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:56.559255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.613913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440c803-bc22-49cd-b567-d4ffda0c4680 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.653831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.654472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.654789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.655061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.664129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.665789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:56.678498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:56.681329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:56.682786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:56.685234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 372/746] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:56.686011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.718151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:56.780079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:56.801042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e37e8-f009-4299-afd1-ccfdcbc17f22 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.809672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca1ea2-5e75-4109-9b91-4d764d6c297a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:56.847407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.848372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.848840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.849283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.850194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.853248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.853801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:56.854523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:56.854964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:56.855294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:56.856263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:56.856588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:56.858433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:56.874589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 373/747] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:56.875373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:56.950649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 375/748] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:55:56 2022] POST /identity/v3/services => generated 240 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:55:56.950649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.039990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7464d-bdb3-4e06-a528-af7f5f216b5a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.090782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.091400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.091736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.092009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.092708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:57.093075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.094402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.096230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.099525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.106566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819e942-ad95-4c8a-9b3f-76c07f8caec9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.109457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 374/749] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:57.110499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.156730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.157657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.158120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.158584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.159480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:57.160056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.161681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.180048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 376/750] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:57.180048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.268054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d2503-d9e9-4ef0-8aaf-c7a685f822b0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.292501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a679-b6c5-49c5-84cb-4062b6ff5c44 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.310256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.311042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.311384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.311732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.319875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.321628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.328992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.329633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.329918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.330197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:57.334497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:57.335453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 375/751] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:57.338065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.339593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.344259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:55:57.352749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:57.354949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:57.357162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 377/752] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:57.366733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.381772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.401348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.447652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1244fb-4ea3-4304-b50a-733bcf204c11 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.477968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f3fd5-36eb-4e84-8bf3-b7f0f4b378de None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.484756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.485376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.485661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.485932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.486691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9d3c0f71e8984988b432390ec99977de) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:57.487006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.488449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.502648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 376/753] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/role_assignments?user.id=9d3c0f71e8984988b432390ec99977de&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:57.502648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.536711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.537439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.537852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.538315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.539439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:57.539823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.541389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9b779-e0a1-4128-b83c-7931b840e0a5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.564567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 378/754] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:57.565825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e3b-dd4b-490d-8797-d9adda4dfd94 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.566114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e3b-dd4b-490d-8797-d9adda4dfd94 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.566389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6e3b-dd4b-490d-8797-d9adda4dfd94 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.570904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 379/755] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:57.572936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6e3b-dd4b-490d-8797-d9adda4dfd94 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.636918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d10c42-ca52-4363-a114-71d14a8428bf None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.666927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.679803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.683459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.684168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.684492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.684747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.685565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:57.685871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.687360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29332715-05d2-4525-ac3e-cff295ccf790 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.709335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 377/756] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:57.710833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.711823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.712228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.716494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6e3b-dd4b-490d-8797-d9adda4dfd94 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.734388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:57.736147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:57.753030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.753678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.753963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.754314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:57.770783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:57.772970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:57.793090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:57.794177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:57.795118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:57.795118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:57.795118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:57.799255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 380/757] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:57.800306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.833562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 378/758] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:55:57.834876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:57.927690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3199b3-1846-4670-bd62-0e23570fd365 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:57.960258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:57.984509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:57.985488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:57.986022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:57.986501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.058953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd719c-1c77-4a7e-818f-c9fbb5b295f7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.072081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.109038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.111276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.112411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.113838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.114359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.115279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:58.117821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.118576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.120420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.139698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 379/759] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:58.147636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.281670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b61dff-d135-4a98-9a61-5b51e01bd3b7 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.289962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 381/760] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:55:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:55:58.290999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.328672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.336739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.337789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.340144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.340144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.348250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.350359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:58.370069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:58.371088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:58.371972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:58.372808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 380/761] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:58.374704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.413526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.418016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09d1bf-59a2-4f81-aa15-f7d57444d6ac None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.465199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.466224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.467309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.467798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.468597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.471027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:58.485153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:58.486228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:55:58.487118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:55:58.493485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 382/762] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:58.493485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.505297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc31ab9-644d-41e1-b20a-311c37c83181 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.551767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.553239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.554000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.554559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.569323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.572596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:55:58.591370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:58.592406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:55:58.593316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:55:58.593316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:55:58.593316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:58.593316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 381/763] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:58.593316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.613589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.624316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb03522-96b6-4576-80e9-27a303d4715b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.662768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.663444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.663726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.664101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.665832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:58.666201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.667693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.687885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 383/764] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:58.687885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.694318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.720528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4e289-731a-4fde-99e5-42f58779ef7d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.760962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.761818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.762173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.762513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.763241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:58.763999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.765059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.770558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 382/765] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:58.771713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.830107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7beb1-d561-4e04-b187-2f55d731265e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.879367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.880211 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.880878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.881290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.882671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:58.884022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.885553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:58.885756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fc2ba-3d6a-477e-9586-18b56a70c756 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:58.902491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 384/766] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:58.904324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:58.919205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:58.941463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:58.942295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:58.942577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:58.942879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:58.946121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=9dd40d8b3cb543feab8424f1d87b7126, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:58.948508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.021829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264d130-f020-409f-b593-26ccad0a3d64 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.060782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 383/767] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:55:58 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:55:59.062708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.073634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.074768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.075509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.075835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.076206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.076875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:59.077188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.078768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20318b1c-1413-4d9f-9845-a5dd760252f9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.085117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 385/768] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:55:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:59.087483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8e33-d55b-4a01-8a31-18efc594dcf3 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.087696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8e33-d55b-4a01-8a31-18efc594dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.088010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8e33-d55b-4a01-8a31-18efc594dcf3 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.089572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 386/769] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:59.094783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b8e33-d55b-4a01-8a31-18efc594dcf3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.209167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200de55b-5ce4-45e0-8531-3d4580f79e44 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.210037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.258654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b8e33-d55b-4a01-8a31-18efc594dcf3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.261190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.261859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.262144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.262417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.271993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.273002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-32fd654f-f5d1-40fd-847b-014cfcfe528c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:55:59.291672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:59.292659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:59.293571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 384/770] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:55:59.294573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.294573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.294763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.313454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.314092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.314205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:59.314368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.314681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.315415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] RBAC: Adding query filter params (interface=public, service_id=75ccfdf30d1a47f0b324010f818f9501, region_id=RegionOne) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:59.315526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:59.315732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.317146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.328798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 387/771] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/endpoints?service_id=75ccfdf30d1a47f0b324010f818f9501&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:59.329960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.391377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.405489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 385/772] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:55:59.406713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.487560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b884fb-c5f2-4d89-b75b-2ab0fd42f790 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.548940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f567df6-deb4-4ae7-801a-e5d4747a90c8 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.556927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.557840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.558337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.558832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.559797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:59.560308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.562087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.589041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 388/773] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:59.589041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.594840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.595375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.596010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.596290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.596565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.597186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.598664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.616364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 386/774] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:59.705051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.773597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef75c-3e00-4815-90e7-f9df8363e5c3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.779419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.812903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.813641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.813921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.814194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.815040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ecc0db48a09441eb68340c620de4263) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:55:59.815350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.816898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e767d1-31b3-49fd-8be6-141579566555 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.823562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 389/775] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/role_assignments?user.id=4ecc0db48a09441eb68340c620de4263&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:55:59.825518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034057-7704-4511-bda6-ef95a2988959 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.825879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034057-7704-4511-bda6-ef95a2988959 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.826347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034057-7704-4511-bda6-ef95a2988959 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.827916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 390/776] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:55:59.830207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097d370-fcef-496b-ac18-bb0533777a0d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:55:59.834171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.834171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.834506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.855485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:55:59.856937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:55:59.874633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:55:59.875512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:55:59.875860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:55:59.876356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] PATH_INFO: `/v3/services/network` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:55:59.877034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:55:59.879870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:55:59.895310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:55:59.896354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:55:59.897661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 387/777] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:55:59.903894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:55:59.936802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:55:59.951417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 391/778] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:55:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:00.072200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697473-64fa-4184-9b66-466b0e323f0a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.094076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:00.108861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.109471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.109752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.110025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.110679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] RBAC: Adding query filter params (name=network) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:00.110985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:00.112368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:00.117956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 388/779] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:55:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:00.123812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.268446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73cbfa-41d0-41c6-b290-b7a7adccdb89 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.306249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.307156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.307692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.308125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.308918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] RBAC: Adding query filter params (type=network) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:00.309375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:00.310935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:00.316785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 392/780] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:00.322976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.330968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.403772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:00.493972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd9cd1-ec11-4f2c-81ab-f281a2cc6569 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.499465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13c619-5fe7-437a-bb14-f189bdbfdd37 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.539298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.540127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.540414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.540692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.565758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:00.570454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:00.571405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:00.572253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:00.574715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.574715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.575394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 393/781] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:00.579456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.579732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.580472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] RBAC: Adding query filter params (interface=public, service_id=f81da545a11b4f3780a0583f3836e92c, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:00.580786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:00.581201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.586641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:00.591558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 389/782] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/endpoints?service_id=f81da545a11b4f3780a0583f3836e92c&interface=public®ion_id=RegionOne => generated 194 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:00.601066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.662644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:00.712167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241afdbe-c49d-4d0c-ab96-694c98e13be2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.750645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.750645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.750645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.750645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.751067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:00.751067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:00.753028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 390/783] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/services => generated 772 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:00.763978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebe1b0-a0a7-418b-bfb7-34c407c983cc None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.798994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:00.832526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:00.833077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:00.833366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:00.833647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:00.834343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:00.834674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:00.836848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:00.851914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 394/784] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:00.855318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:00.982181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd855-10c9-4df9-9750-831871a2f7ac None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:00.997909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:01.020374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:01.021352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.021855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.022332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.034650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:01.036441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:01.051783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:01.052742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:01.053642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:01.053642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:01.053642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:01.056313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 391/785] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:01.062232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:01.169950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91794007-e51e-42e1-8a82-b377bf6467f5 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:01.212575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:01.213574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.214075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.214562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.215458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:01.215995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:01.217623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e721f-673b-4b7d-867e-fc2a39cfc72c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:01.235459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 395/786] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:01.241127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:01.341575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9fcc-f666-462a-8e7f-25e19b92d8a2 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.342221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9fcc-f666-462a-8e7f-25e19b92d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.342725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9fcc-f666-462a-8e7f-25e19b92d8a2 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.344907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 396/787] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:01.349766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.350283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.350788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.368707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb2759-027f-4dba-a6a5-fed7fc4bb818 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:01.372267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:01.373702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:01.416402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:01.443378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:01.443938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.444219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.444488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.459587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:01.462266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:01.470326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 397/788] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:01.482070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:01.483057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:56:01.483999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:01.486526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 392/789] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:01.495217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:01.631687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b1b0a-5130-4037-9e3a-8f7f1ecd1cb4 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:01.682988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:01.683616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.683903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.684181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.684930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:01.685240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:01.686710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:01.705350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 398/790] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:56:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:01.710394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:01.753319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:01.845116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b1cbb-9b0b-41bc-bb2b-ee587bbeb008 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:01.881366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:01.882377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.883486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.883990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.896388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:01.902643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8a2afd44-210f-4535-a73e-632887867a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:01.914307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:01.920451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:01.921429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:01.921429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:01.921429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:01.921429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 393/791] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:01.924390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:01.936702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced4de-b18d-4e27-ba15-dcad01edcdcb None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.936978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced4de-b18d-4e27-ba15-dcad01edcdcb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.937247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ced4de-b18d-4e27-ba15-dcad01edcdcb None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.938573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 394/792] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:01.950323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:01.950577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:01.950922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:01.963547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.054214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aefc3-b667-474e-a3b4-a674ecfc75ed None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.094632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d664d-2803-4b91-940d-b74111936ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.095612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d664d-2803-4b91-940d-b74111936ead None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.096105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d664d-2803-4b91-940d-b74111936ead None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.096622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d664d-2803-4b91-940d-b74111936ead None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.097487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d664d-2803-4b91-940d-b74111936ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:02.098011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d664d-2803-4b91-940d-b74111936ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.099516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:02.099875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269d664d-2803-4b91-940d-b74111936ead None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.100726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:02.112774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 399/793] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:02.114084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d664d-2803-4b91-940d-b74111936ead None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.178756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 395/794] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:02.179744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.197114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.209819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.228107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269d664d-2803-4b91-940d-b74111936ead None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.273731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.275274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.275768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.276251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.287837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:02.293857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:02.294824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:02.295672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:02.298290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 400/795] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:02.326662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.342408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4633d71-a360-406c-a938-73e9d96fe2c2 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.377521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.378164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.378454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.378728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.385777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.387400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-549b36bd-e843-41b8-a5ab-32ac0aabae58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:02.399507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:02.400694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:02.401972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 396/796] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:02.402751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661e01-d2ea-4be9-92a1-0bb11c62e743 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.402751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661e01-d2ea-4be9-92a1-0bb11c62e743 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.402993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661e01-d2ea-4be9-92a1-0bb11c62e743 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.404414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 397/797] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:02.406700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24661e01-d2ea-4be9-92a1-0bb11c62e743 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.475835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05da64-d256-4990-be37-667eca2aa546 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.511334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.511863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.512147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.512252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24661e01-d2ea-4be9-92a1-0bb11c62e743 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.512413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.513087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:02.513392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.513905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.514806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d26174-aa3e-4a61-b530-a7657775184e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.528814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 401/798] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:02.530135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.530425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.530729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.549261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.549742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:02.549918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.550202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.550477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.550968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:02.551250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:02.551608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.552941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.553221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.570177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 398/799] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:02.570177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.629910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 402/800] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:02.635873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.693127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02855b7-2725-4abb-9c8f-d0d671f7421d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.726675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.737335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.737974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.741033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.741033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.751837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.753466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:02.772642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:02.774472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:02.776338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:02.776338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:02.776338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:02.776338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 399/801] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:02.776338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.778031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18cd6b-9b9b-48ca-866f-a32539602412 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.828240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.829735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.830492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.831389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9d3c0f71e8984988b432390ec99977de/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.834845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=9d3c0f71e8984988b432390ec99977de, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.837720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.900483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bdf3-cf58-40d9-9534-c42ac88ee530 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:02.906150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:02.915613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 403/802] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:02 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9d3c0f71e8984988b432390ec99977de/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:56:02.917007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:02.949513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:02.950186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:02.950467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:02.950773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:02.951725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:02.953402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:02.966415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:02.967550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:02.974006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 400/803] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:02.974888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.025444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.060697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac379762-7957-49e4-aeaa-53db41e1aed4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.089193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f878c-93fb-4d11-ad17-ec9a2f8c4117 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.099199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.099815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.100095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.100368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.101017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:03.101373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.102785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.115802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 404/804] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:03.117390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.127299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.127923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.128217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.128514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.139931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-45831adb-540b-4ade-9830-cad607eabb63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:03.145034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:03.145957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:03.146815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 401/805] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:56:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:03.147559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223947-a20e-47f7-83e1-2acc56e54cad None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.149088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223947-a20e-47f7-83e1-2acc56e54cad None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.149088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223947-a20e-47f7-83e1-2acc56e54cad None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.150409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 402/806] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:03.156812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30223947-a20e-47f7-83e1-2acc56e54cad None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.189661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.233012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1db4bc-a229-429c-b0d0-ab26707031db None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.261441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30223947-a20e-47f7-83e1-2acc56e54cad None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.263045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.269880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.270583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.270898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.271175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.271815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:03.272136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.273556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.283282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 405/807] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:03.283282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.304047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.305931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.306212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.306481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.314089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.315561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-24045f5b-9179-49e2-a93d-9ab275914e03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:03.328641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:03.334560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 403/808] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.335866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.336909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.352201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:03.353386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:03.395398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52f766-77b6-40c6-9138-83d5a6d06056 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.420396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.428635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 404/809] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:03.430117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.442975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.443852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.444329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.444785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.446427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:03.446924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.448578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.463853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 406/810] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:03.465100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.538588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.548337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a4331-f587-4a0b-b3b5-4f252232f623 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.595569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.596708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.597540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.598085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.599022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:03.599570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.600395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e1200-6a95-4939-a0c4-e2aa2bf74468 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.602160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.615367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 405/811] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:03.616489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.645837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.646946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.647324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.647707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.649399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:03.650074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.651996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.686449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 407/812] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:03.686449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.736846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.760504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e7db1-af84-4957-a678-5a7e28223a17 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.803945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.804620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.804963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.805299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.811615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80740bd-96f3-454a-8ec7-959f4218831f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:03.816431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.817905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:03.834972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:03.837230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:03.839053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:03.839053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:03.839053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:03.839053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 406/813] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:03.839053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.853666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:03.857551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:03.858166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:03.858536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:03.859042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:03.859951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:03.862176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:03.963475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 408/814] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:03.963475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:03.976946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910db45-5771-4db6-8352-98803b79ae2d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.047168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.047905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.048264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.048544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.137401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.145093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb8a9-579d-43fb-b80e-f2966214c6c1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.147902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.150664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.153337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.213336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.215086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.218640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.218640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.218640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.218640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:04.228012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:04.229302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 05:56:04.230274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:04.232523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 409/815] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:04.232523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.311330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 407/816] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:04.312971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.360522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f5834-93f4-414a-8beb-50fdab80fef0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.405421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.405966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.406256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.406535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.407236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:04.407553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.408970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.423218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 410/817] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:04.424165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.450367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7b15-2d5b-480b-8bd7-f64011407f30 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.451974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.453539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.491079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.493626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.493952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.494234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.494988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] RBAC: Adding query filter params (name=network) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:04.500441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.500441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.508692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 408/818] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:04.508692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.551005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa55fc-7acd-4d3c-bf41-67216872c931 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.590741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.591388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.591664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.591938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.603489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.604958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:04.622097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:04.623169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:04.624053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:04.624053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:04.624053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:04.626885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 411/819] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:04.628110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.631371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78b6b5-45c2-4016-a51a-6a376032d443 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.672403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.673641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.674145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.674646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.684183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.686087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:04.700451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:04.701466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:04.702511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:04.705294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 409/820] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:04.706474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.718940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.741031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.748133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad535d-341d-4bf2-9b88-8a09c04992bf None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.785096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.785707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.785993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.786268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.786961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] RBAC: Adding query filter params (type=network) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:04.787289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.790552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.800801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 412/821] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:04.800801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.835300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc61ffc-ee7a-459c-9c18-a4c87a4bf7ab None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.876576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.877655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.878224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.878728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.879573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:04.880061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.881643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.895981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 410/822] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:04.897648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:04.925029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e86f6-e1ac-405d-995d-50a2ceae677b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:04.962747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:04.963363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:04.963644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:04.963913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:04.964648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:04.964949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:04.966425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:04.977404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:04.990748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 413/823] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:04.991598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.000755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.015605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75a5f7-ee71-4ee0-95e0-68b21354a6fb None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.053951 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.054647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.054965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.055205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.055869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.057235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3663847-fffc-4e1a-a2d5-b030d21e7c02 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.102844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abfc3c-84db-491a-96b7-4cc61c020c65 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.137836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.138480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.138798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.139077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.146030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.147542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.154078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 411/824] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:04 2022] POST /identity/v3/endpoints => generated 337 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:05.156359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aecd19-45f8-493f-ae85-1ef0ccec4467 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.157091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aecd19-45f8-493f-ae85-1ef0ccec4467 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.157599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aecd19-45f8-493f-ae85-1ef0ccec4467 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:05.163465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:05.164790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:05.165683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:05.166699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 414/825] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:05.172024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 412/826] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:05.172167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.172246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.172246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.172246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.196388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:05.198164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:05.238044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.296320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3df6f-5cc2-4d73-ba4d-612a1cc55f25 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.334073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 413/827] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:05.335585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.346287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.347495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.347953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.348455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.361807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.363974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:05.381378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:05.383153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:05.384044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:05.384044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:05.384044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:05.386190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 415/828] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:05.398694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.476828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.529536 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e135dd-0c2e-4528-b417-5c2184ef0c97 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.547873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf8fdc-8e90-4eee-aebb-ff6667e5577d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.561494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.570717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.571637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.572064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.572485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.574581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:05.574581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.575192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.589269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 414/829] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:05.593863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.594482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.594803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.595079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.595731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:05.596034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.597668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.598909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.613559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 416/830] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:05.622722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.722458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059aa0c-608a-4e75-976b-6f1bc7074735 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.732668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083911c-f04a-463b-a69d-224be8b50d4b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.756684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.763561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.764171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.764458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.776002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.776002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:05.776002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.776002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.780831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 415/831] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:05.781952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.792509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.794317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.794915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.795387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.807103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:05.809229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:05.826665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:05.827634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:05.828559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:05.836441 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 417/832] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:05.838817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:05.881957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:05.931892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70db41-5033-41a3-b42e-605b5446a6bd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:05.982857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:05.983550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:05.984050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:05.984323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:05.993709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e6685-432d-4073-b763-31f120fb369f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.000782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:06.008214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:06.010058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:06.011466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 416/833] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:06.016025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.038788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.039429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.039880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.039988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.040635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:06.041743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.042450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.056329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 418/834] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:06.062089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.081088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:06.143784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16beb42-4afc-41ed-bbcc-a2bd60d6c35a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.188006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:06.199780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.200951 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.201412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.201834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.202477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139c7a-487a-4d15-88bb-cc2bd593cc4a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.202910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:06.203647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.207850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.219978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 417/835] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:06.225257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.256663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.258860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.259430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.260209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ecc0db48a09441eb68340c620de4263/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.277309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=4ecc0db48a09441eb68340c620de4263, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.279103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7015fa3-b4b2-4396-81b5-997e8754f927 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.358118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20d61d-0cbf-4a9e-a4d0-1b3904da01a0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.369801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 419/836] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:06 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ecc0db48a09441eb68340c620de4263/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:56:06.372300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102fa1-9bae-4ab6-ae96-ba62d25f3e58 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.373901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102fa1-9bae-4ab6-ae96-ba62d25f3e58 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.373901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c102fa1-9bae-4ab6-ae96-ba62d25f3e58 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.376244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 420/837] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:06.382842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.383232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.383581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.391407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:06.406254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:06.408104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:06.415853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.416757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.417140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.417515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.431763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.433733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:06.452906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:06.454126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:06.455086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:06.455086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:06.455086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:06.455728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 418/838] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:06.461273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.513066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 421/839] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:06.590460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca9582-54a9-46c3-a151-843989d8a615 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.636341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.636959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.637291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.637609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.638368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:06.638699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.640183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.654387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 419/840] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:06.659741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.734387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:06.768829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fb29a-b443-4e8e-9b1d-1fad12b85cba None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.793053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.816307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:06.816957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:06.817269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:06.817550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:06.829830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:06.830422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:06.833323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:06.854218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:06.855657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 05:56:06.857101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:06.860230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 422/841] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:06.861370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:06.954680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeba52-6667-45ad-a7fd-5f65a9e774aa None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:06.989546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f915f-e928-402e-9a0e-98cf019c57f1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.004443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.005928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.006695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.007157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.009080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.011069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.020276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:07.026246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:07.033248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:07.034232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 420/842] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:07.036187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.036657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.037306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.037590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.037868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.039252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:07.039569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.041053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.068939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 423/843] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Aug 26 05:56:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:07.075700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.166331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.177206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6703-6c1f-4c23-8402-a40ff371fc21 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.187103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581fa6-81f3-4cbc-b342-913a3a9f5656 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.216531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.217608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.218174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.218707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.219548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:07.220164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.221838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.226985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.227735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.228000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.228274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.229362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 421/844] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:07.235813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.308636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.326942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.328508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.362063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a15787-8c6d-4729-8935-d7821dfd2b79 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.401892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.403071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.403639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.404175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.405034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:07.405545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.407299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.412959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 422/845] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:07.451662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 424/846] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:07.457146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.487036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.580554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa0d17-0562-4776-b259-977d275f1692 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.617938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.618591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.618905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.619180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.631093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.633186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:07.648248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:07.649407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:07.650546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:07.650546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:07.650546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:07.650546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 423/847] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:07.657084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.749674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.765407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee419d-87a2-41f6-80ce-d13ad9d7e457 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.806687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.807763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.808282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.808762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.809624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:07.810136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:07.811843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0606d-59d9-4b5d-b4f1-4aba9ee01dec None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:07.826342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 425/848] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:07.834760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:07.917865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:07.955055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1153fec-516b-4ba8-945a-de7674cd37f9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:07.970101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0df81-c90f-4236-afc1-c608f07a75d4 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.970437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0df81-c90f-4236-afc1-c608f07a75d4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.970700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0df81-c90f-4236-afc1-c608f07a75d4 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.972325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 426/849] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:07.978846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.979344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.979639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:07.996690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:07.997436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:07.997997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:07.998281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.002100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:08.003420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:08.006126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:08.008821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:08.025767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:08.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:08.027653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:08.028107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 424/850] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:08.034682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.103881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 427/851] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:08.174327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5a093-51da-42cf-afed-f0a1e0f39f02 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:08.211085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:08.216379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:08.217083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.217427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.217763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.218546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:08.218942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:08.220623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:08.236608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 425/852] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:08.243405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.345694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:08.392571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68049bd3-c01e-4052-85b8-0c71cc9f43c1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:08.419567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.452025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:08.452815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.453102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.453380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.454646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9d3c0f71e8984988b432390ec99977de) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:08.455136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:08.457319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698f8e1-91b6-43de-8691-5d95d9354396 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:08.466191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 428/853] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/role_assignments?user.id=9d3c0f71e8984988b432390ec99977de&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:08.514787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fbdc1-af78-4589-b671-86fdc0580397 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.515106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fbdc1-af78-4589-b671-86fdc0580397 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.515352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fbdc1-af78-4589-b671-86fdc0580397 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.517641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 429/854] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:08.523711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.524020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.524355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.548809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:08.550875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:08.553825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda89f5-10f8-461a-8ba6-bd84c55f86d4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:08.621418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:08.623015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.623300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.623580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.624277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:08.626398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:08.640655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:08.641654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:08.643882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:08.646912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 426/855] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:08.648610 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.663027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 430/856] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:08.727577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:08.790647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c81c3-3ad6-4775-9fa4-e773c3b0c405 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:08.832569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:08.833189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:08.833521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:08.833800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:08.834443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:08.834781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:08.836968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:08.842472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 427/857] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:08.849451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.938243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:08.956399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:08.974810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892d656-ca26-4c7d-af0e-86f52fd58009 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.024078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.024969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.025264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.025568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.026741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:09.027060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:09.029024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:09.035919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 431/858] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:09.090681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634633d-70ab-41bd-a2ff-5b06a0e47069 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.119931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:09.136376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.146006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.146596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.147048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.160423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:09.166579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:09.167759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:56:09.168783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:09.171224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 428/859] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:56:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:09.177417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:09.287362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f68594-f19e-49d0-a369-ca5e165c52a2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.325128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.326135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.326758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.327429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.328285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:09.328788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:09.330416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:09.332084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:09.340643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 432/860] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:56:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:09.346320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:09.453338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b714d9-c253-4879-aca2-44dcd8b2c625 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.489286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.490362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.490923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.491548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.504880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:09.506792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-75e13fbf-d748-40fc-8021-64db7cafda8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:09.522537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:09.524187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:09.525414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:09.525414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:09.525414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:09.527272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 429/861] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:09.533265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:09.643952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b711-652f-4a51-a9ca-3327b0945ae8 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.657481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c4a5-b61b-40e9-a44d-99fdad12e2a4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.657764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c4a5-b61b-40e9-a44d-99fdad12e2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.658039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c4a5-b61b-40e9-a44d-99fdad12e2a4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.660620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 430/862] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:09.665014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.665300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.665594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.681169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.681801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.682094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.682379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.683071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:09.683386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:09.684750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:09.684870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:09.685940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:09.697867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 433/863] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:09.704469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:09.719672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:09.778426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 431/864] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:09.787026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:09.829969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf784b2-b5d2-4c97-a84f-6e2bbaa600c6 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:09.865663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:09.867245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:09.867803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:09.868328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:09.875983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:09.877743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:09.889936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:09.890916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:09.891864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:09.894342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 434/865] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:56:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:09.900663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.004361 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb20bc-b0bc-4889-aaa2-e6453e7d335b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.040337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.040950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.041228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.041501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.042231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:10.042532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:10.043992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:10.055866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.064203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 432/866] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:56:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:10.069797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.091662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:10.171747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb16bae-cbc9-4b43-93df-fc09ab418535 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.173068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816dda7-1c2f-4876-9402-3004ce0f1f90 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.189690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:10.208877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.209505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.209787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.209897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.210077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.210712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.210996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.211323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.211994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:10.213419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:10.285999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:10.287820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:10.290920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 435/867] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:10 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:10.315628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:10.407282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 433/868] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 05:56:10.413369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.539374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2407b7-ad0f-474e-af5a-98dc3cd91ffd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.579375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.580468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.580910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.581331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.594898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:10.596594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-414bc2ae-ad98-40c5-965e-9d38bfd760d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:10.612756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:10.613944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:10.615076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:10.615076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:10.615076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:10.616847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 436/869] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:10.622895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.663866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867020a-5a9f-4891-b34b-00245e867cbb None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.664139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867020a-5a9f-4891-b34b-00245e867cbb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.664414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867020a-5a9f-4891-b34b-00245e867cbb None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.665891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 437/870] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:10.672382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.672693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.672941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.692029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:10.693398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:10.716158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:10.743944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf6a3a-8d61-4664-bb57-034cca84173b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.769799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:10.788172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.789538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.790000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.790447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:10.791388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:10.796105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:10.796105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:10.806719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 438/871] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:10.815217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 434/872] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:10.824038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:10.952685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342c008-b11d-489d-b4c2-28cef4ed8f0d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:10.994504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:10.995189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:10.995478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:10.995818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.003040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.004576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1f981b0d-5b55-4db7-8dda-376278cdd710 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:11.016591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:11.017560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:11.018497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 439/873] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:11.025106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.119548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed31db-8147-43aa-98b6-26cd6473fd42 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.119889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed31db-8147-43aa-98b6-26cd6473fd42 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.120313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed31db-8147-43aa-98b6-26cd6473fd42 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.122817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 440/874] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:11.129146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.129436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.129712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.155981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:11.157951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:11.171180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:11.183198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc76d-9eec-4b29-b139-1ee660c765b3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.226499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.227239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.227533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.227809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.228513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:11.228818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.230301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.239961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 441/875] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:11.248448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 435/876] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:11.249680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.249809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.297989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:11.367567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9f8-f46e-4744-aa9c-4fe1fdc78157 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.402934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394f0e2-7c08-48f1-a0dd-c82d053e05f9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.418644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.419344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.419629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.419911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.420861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ecc0db48a09441eb68340c620de4263) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:11.421405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.422966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.430658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 436/877] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:11 2022] GET /identity/v3/role_assignments?user.id=4ecc0db48a09441eb68340c620de4263&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:11.454181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.455220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.455789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.456254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.457174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.458875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:11.470330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:11.471602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:11.472450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:11.474797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 442/878] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:11.487831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.522162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.534970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:11.636884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022cfd7-0b9c-4037-ab1c-8f153dd20e81 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.641625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:11.680918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.682053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.682550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.683089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.683964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:11.684472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.685890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.691384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 437/879] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:11.693288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3008b-d2b2-4e19-8aac-b5ce72adc8da None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.700642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:11.748090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.749467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.750236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.750717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.751472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.752919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c3729-3ba3-4b32-bf04-fb3bbbb078ef None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.834301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821143aa-cf9c-4e9f-839b-57c94040a98f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:11.870973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 443/880] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:11 2022] POST /identity/v3/services => generated 269 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:11.896441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:11.897097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:11.897429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:11.897834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:11.898659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:11.898978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:11.900692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1e0e-779e-4bf0-bec0-42606833a3c8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:11.907053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 438/881] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:12.475558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f934-64ef-46dd-b1c5-94e077095bbb None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:12.476653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f934-64ef-46dd-b1c5-94e077095bbb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:12.477104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f934-64ef-46dd-b1c5-94e077095bbb None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:12.478903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 444/882] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:12.485425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:12.486339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:12.486864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:12.506389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:12.508402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:12.535485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:12.590285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 439/883] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:12.869389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f934-64ef-46dd-b1c5-94e077095bbb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:12.947097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:12.985338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f934-64ef-46dd-b1c5-94e077095bbb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:13.022400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:13.023846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.024312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.024842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.025452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:13.026890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:13.036811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:13.037873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:13.039178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 445/884] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:13.044622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:13.167040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfe98c-e750-4d70-95f0-b7d312c9fad3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:13.202687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:13.203696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.204200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.204692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.205542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:13.206045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:13.207656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:13.213216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 440/885] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:13.222658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:13.323636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2869e-4b89-4d31-b288-6ecfa1ee879b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:13.354794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:13.359873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:13.360501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.360786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.361060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.361701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:13.362074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:13.363526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:13.369044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 446/886] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:13.375269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:13.401685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:13.479837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be93324-8e9c-47f2-96ad-dd60a165c807 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:13.515544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:13.516936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.517460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.517979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.518927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] RBAC: Adding query filter params (interface=public, service_id=6eff8ac3f4d74ca7b7031e39846ad0d4, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:13.519443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:13.521022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1f77c-56c1-462f-b9f9-8e318187ee58 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:13.527900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 441/887] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:13 2022] GET /identity/v3/endpoints?service_id=6eff8ac3f4d74ca7b7031e39846ad0d4&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:13.538862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:13.641216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65020e7d-9063-4f26-b2bc-6729687fbb19 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:13.677907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:13.678526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.678897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.679187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.679888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:13.681266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3800d-aff1-4a1d-8394-45ce49bd09da None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:13.687898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 447/888] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:13 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 05:56:13.688760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0f59b-0b44-40f9-b27e-86d3cccda384 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.689276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0f59b-0b44-40f9-b27e-86d3cccda384 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.689766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0f59b-0b44-40f9-b27e-86d3cccda384 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.691638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 442/889] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:13.698014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7083-c1e5-4968-89f9-db66bd4b54e6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.698303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7083-c1e5-4968-89f9-db66bd4b54e6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.698588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b7083-c1e5-4968-89f9-db66bd4b54e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.719249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e96b7083-c1e5-4968-89f9-db66bd4b54e6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:13.720556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e96b7083-c1e5-4968-89f9-db66bd4b54e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:13.786095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:13.798703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 448/890] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:13.972844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e511-e60d-4743-8638-f0dd948ce52d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.973499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e511-e60d-4743-8638-f0dd948ce52d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.974002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7e511-e60d-4743-8638-f0dd948ce52d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.976401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 443/891] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:13.981744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.982214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.982399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.988148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceca16-90b0-499c-b2bc-ddeaf88574d0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.988438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceca16-90b0-499c-b2bc-ddeaf88574d0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.988716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceca16-90b0-499c-b2bc-ddeaf88574d0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:13.990210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 444/892] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:13.996363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:13.996652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:13.996926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.002635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:14.004089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:14.015178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:14.016685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:14.077613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 449/893] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:14.093303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 445/894] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:14.103006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.206377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca7fa9-e044-44d8-bcd4-19ef2f972eca None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.233003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.248010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.250402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.250402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.250402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.261302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:14.266915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:14.267887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:14.268766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 450/895] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:14.276274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.295004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.373994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.384678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b62b8-2400-443a-a131-e79f44cff665 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.421141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.421748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.422031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.422308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.423007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:14.423313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:14.424689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:14.437641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 446/896] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:14.437641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.487112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d9adc-a9be-411c-95a3-40b7b325d003 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.524199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.525054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.525338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.525609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.526265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:14.527507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.527969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:14.549051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada93a82-5032-415c-bb6b-bbe33fcae4e9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.583582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.584497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.584980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.585976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.586761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:14.588293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:14.591004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 451/897] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:14 2022] POST /identity/v3/services => generated 241 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:56:14.591842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:14.598450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:14.600477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:14.601706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:14.608149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 447/898] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:14.609434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.662983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.754863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a5766-747f-47de-a418-83b5b7447d19 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.755102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718b4fd-76cd-4490-90b5-035b96f17d00 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.808010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.808761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.809115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.810403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.817395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:14.817919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:14.818353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:14.818547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:14.819171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:14.820254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:14.821669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:14.827479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:14.829917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:14.830717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 448/899] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:14.837627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:14.852741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:14.859931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:14.860813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:14.860813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:14.860813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:14.861068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.861141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 452/900] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:14.861141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:14.974084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:14.982763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758349-c0c9-43c1-8320-f2d4aad6453c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:14.994437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934cbbe-c6e0-42ad-b649-829c36a40610 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.024150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.025198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.025568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.025936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.026740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:15.027144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.028830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.039088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 449/901] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:15.041416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.047414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.047833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.048124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.048400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.049057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:15.049359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.050793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.070767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 453/902] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:15.072694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.138981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:15.184960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d2b2-36a9-4d03-9537-546e4e081111 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.197629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e36d7-2551-469d-9d6b-25d9c0e52707 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.234392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.235076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.235356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.235914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.237059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] RBAC: Adding query filter params (interface=public, service_id=006b4537ed944dcfb064c79966c08dcb, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:15.237372 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.239345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.245731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.246466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.246862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.246966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 450/903] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/endpoints?service_id=006b4537ed944dcfb064c79966c08dcb&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:15.247142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.252667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.260827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.262527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:15.279175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:15.280171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:15.281093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 454/904] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:15.286017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.302838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:15.375751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a43d7a-0a69-438d-9420-399be303bcb7 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.416258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.417356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.417909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.418413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.419301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.419734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b2377-eb94-45ea-bae1-4db034e76b89 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.421542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.423009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:15.438856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 451/905] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 05:56:15.460847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.461474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.461757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.462031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.462812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:15.463271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.464753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b96054-6b80-4aa9-a9dc-c9be7824fc17 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.485820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 455/906] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:15.492369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.609032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6b902-f0be-4915-849d-da2c1bb1f064 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:15.652216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:15.652838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.653120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.653393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.708548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8251e6-3966-4570-9746-bb99659c0b84 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.708871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8251e6-3966-4570-9746-bb99659c0b84 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.709119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8251e6-3966-4570-9746-bb99659c0b84 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.711860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 456/907] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:15.717109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:15.717401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:15.717685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:15.732996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:15.736887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:15.738301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:15.738541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:15.740086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:15.820623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:15.835176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 457/908] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:15.869652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 452/909] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:15.875224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:15.988446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781edda7-5735-4835-b42b-3d45ff48494e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.027860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.028486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.028786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.029084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.041476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.043363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:16.061174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:16.062177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:16.063097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:16.063097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:16.063097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:16.063097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 458/910] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:16.069027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.154797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.181778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.186168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba7b236-704a-468c-8662-e5925da625c4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.221132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.224608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.225510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.226550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.227014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.227837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:16.228290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.229834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.247394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 453/911] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:16.252904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.295683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222414b9-69f0-4bf6-ae74-120db7cff7f3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.333759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.334767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.335778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.336236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.337017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.338586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:16.347798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:16.348855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 05:56:16.349681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:16.353081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 459/912] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:16.358424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.366433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1cd22-201c-4834-bd27-c020ed0cd9d4 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.403960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.404643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.404925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.405197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.412194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.414522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:16.426113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:16.427070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:16.427968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:16.432513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 454/913] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:16.433335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.453952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.471371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe9335-b5d7-43ab-a2a6-ed5a9ebe5a14 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.486151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.509228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.510170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.510685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.511164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.511996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:16.512500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.514081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.519571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 460/914] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:16.530673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.545090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1336e6-b662-4069-97bb-7e7817568b0f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.581623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.582227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.582742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.582888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.583473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:16.583774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.585901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.604069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 455/915] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:16.606004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.646187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7a78f-b92d-4a9a-b2bd-ad600a4505d3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.685262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.686247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.686748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.687175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.688011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:16.688544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.689891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.691607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.703204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 461/916] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:16.704519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:16.715845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dffc22-47f4-4941-86f9-35349a45b347 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.750792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:16.753537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.754123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.754403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.754703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.755448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] RBAC: Adding query filter params (role.id=8788f0cdca9e477cbbcee427c9898dcd, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ecc0db48a09441eb68340c620de4263) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:16.756046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.757472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25a595-2ef2-489f-b348-aad0809ef282 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.770868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 456/917] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:16 2022] GET /identity/v3/role_assignments?user.id=4ecc0db48a09441eb68340c620de4263&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=8788f0cdca9e477cbbcee427c9898dcd => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:16.770868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837440-19b8-4b91-91c9-5abc77188892 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.770868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837440-19b8-4b91-91c9-5abc77188892 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.770868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72837440-19b8-4b91-91c9-5abc77188892 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.770868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 457/918] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:16.772594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.772886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.773647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.807347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:16.824226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:16.831099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ad381-d1a5-4e31-89ad-068fbd070432 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:16.872317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:16.873348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:16.873830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:16.874350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:16.875340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:16.877261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:16.906350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 458/919] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:16.964583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 462/920] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:17.187371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:17.269339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:17.308502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca997b4-9942-45e3-8e6b-55c8449455a7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:17.358793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:17.359439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.359715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.359984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.360569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:17.361942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:17.372446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:17.373480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:17.374577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 459/921] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:17.379903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:17.488525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:17.505079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe380d4-b335-409d-89d7-1a6688bd3cb7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:17.543443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:17.544435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.544915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.545488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.546325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:17.547039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:17.548737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81b271-bebe-476f-9061-ca1003e021a4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:17.555569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 463/922] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:17.561778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:17.639095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c454-bc9c-4154-9f31-004b4d5151ec None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.639764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c454-bc9c-4154-9f31-004b4d5151ec None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.640263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494c454-bc9c-4154-9f31-004b4d5151ec None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.642085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 464/923] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:17.647047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.647364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.648185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.666612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:17.667871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:17.679227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dbe8a-62f4-4555-b5e2-f79de9718e24 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:17.713538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:17.718791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:17.719457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.719735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.720055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.720695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:17.720999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:17.722475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:17.728719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 460/924] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:17.734826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:17.762799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 465/925] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:17.850319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdb36a-3457-4b1e-a19c-e78d1935bd00 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:17.888514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:17.889417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:17.889868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:17.890292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:17.891205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] RBAC: Adding query filter params (interface=public, service_id=8973a0dfa13f49b496fd1d29d7cbd7f4, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:17.891665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:17.893198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:17.900516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 461/926] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:17 2022] GET /identity/v3/endpoints?service_id=8973a0dfa13f49b496fd1d29d7cbd7f4&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:17.906138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:18.007600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:18.017720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b0453-888d-4e0b-979e-2d18d9748697 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:18.055968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:18.056596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:18.056879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:18.057157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:18.057837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:18.059396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:18.059885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:18.066461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 466/927] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:17 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 05:56:18.137549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:18.199527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2a80f-2391-4021-8653-0e0975229600 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:18.254023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:18.255065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:18.255489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:18.255898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:18.257412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:18.259625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:18.275105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:18.276060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 05:56:18.276948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:18.277677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 462/928] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Aug 26 05:56:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:18.286031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:18.401545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62469af9-8894-4a4b-9051-d69c6f71bcf5 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:18.439993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:18.441029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:18.441550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:18.442025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:18.442974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:18.443487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:18.446333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:18.452496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 467/929] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:18.458603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:18.576896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbbf0f-52e2-469c-9001-523940d517ed None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:18.611450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:18.615377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:18.615899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:18.617102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:18.617408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:18.618138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:18.618499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:18.619930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69afc7-8b82-4431-937d-f05279e8519c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:18.625609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 463/930] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:18.631818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:18.656619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:18.755097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7aaa8-ed34-448c-a5b6-196caa50526c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:18.794147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:18.794803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:18.795093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:18.795370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:18.796044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:18.797439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980391fc-35e1-4e6c-bb36-9a02ae9a63e1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:18.924398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 468/931] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:56:19.069395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b776c-c4da-48b6-8962-aa7d6775020d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.070572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b776c-c4da-48b6-8962-aa7d6775020d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.071106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b776c-c4da-48b6-8962-aa7d6775020d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.073091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 464/932] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:19.078831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.079391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.079869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.099446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:19.101733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:19.180788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227855-f86a-421a-8bf0-be87bcdb3bbe None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.181015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227855-f86a-421a-8bf0-be87bcdb3bbe None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.181098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70227855-f86a-421a-8bf0-be87bcdb3bbe None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.183863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 465/933] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:19.186981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:19.188776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.189117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.189387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.190384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 469/934] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:19.210802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:19.212190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:19.298791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 466/935] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:19.469821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:19.579709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:19.592404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8e5f2-3f57-4da9-a334-683cd1cec33a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:19.636864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:19.638121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.639144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.639655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.660972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:19.668065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:19.669208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:19.670252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:19.679325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 470/936] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:19.680453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:19.712597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2965fc-216b-481d-9881-dd4756160e85 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:19.743244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:19.765974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:19.768202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:19.769067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.769477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.769890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.771134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:19.773309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:19.777021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING keystone.server.flask.common [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 05:56:19.816495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da056f-f5eb-4074-a0a4-87ddd48d552c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:19.826110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.sql.core [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 05:56:19.826110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 05:56:19.826110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [parameters: {'id': '65bd50aa109e43bfa0018cdba600ecc2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 05:56:19.826110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81762) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65bd50aa109e43bfa0018cdba600ecc2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65bd50aa109e43bfa0018cdba600ecc2. Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:56:19.838397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result.read() Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 05:56:19.839299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application yield resource Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise self.value Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application session.commit() Aug 26 05:56:19.840262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.session.flush() Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application compat.raise_( Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 05:56:19.841163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 05:56:19.842128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application result.read() Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:56:19.844605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [parameters: {'id': '65bd50aa109e43bfa0018cdba600ecc2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:19.846813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:19.848837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65bd50aa109e43bfa0018cdba600ecc2. Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:19.851693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 467/937] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Aug 26 05:56:19 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:19.853011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:19.867882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:19.868481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:19.868823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:19.869105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:19.869833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:19.870146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:19.871750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:19.892123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 471/938] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:19.894903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.003608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474f1e9-0490-40bd-af1e-2805bc3ba5d9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.021117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0d615-4620-454b-a9c8-dbd324bacb64 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.046530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.047272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.048198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.048486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.051110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.059251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.061583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.067143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.072300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.072300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.072300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:20.075503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:20.076952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:20.078046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:20.082157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 468/939] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:20.083096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.086040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.089003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:20.109817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:20.110992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:20.111912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:20.111912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:20.111912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:20.114696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 472/940] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:20.118408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.131010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.211785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfda7d8-b16d-4e0b-868b-728099ba5a95 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.255456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.256086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.256367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.256677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.257887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:20.258190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.258658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a48de-fc71-49eb-a58a-78c13c463507 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.259662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1bcebf-6516-4eb8-9753-87ce10f77414 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.281258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 469/941] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:20.282948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1780c-ab05-408a-833f-a7b2e165f33f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.283485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1780c-ab05-408a-833f-a7b2e165f33f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.284007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1780c-ab05-408a-833f-a7b2e165f33f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.285798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 470/942] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:20.294781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.294781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.294781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.300346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.300981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.301268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.301550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.302235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:20.302554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.304842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.314015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:20.315694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:20.320131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 473/943] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:20.325951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.370318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.406529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 471/944] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:20.451918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede894cf-03a9-4cfc-b720-6203e1794d81 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.490039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.490796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.491051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.491853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.500115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.501666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:20.514894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:20.516876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:20.518046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 474/945] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:20.525429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.553101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.633595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c451-9ad0-46f5-8793-2ad745e0f586 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.672657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.673276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.673556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.673827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.674688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:20.676301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.676492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.676658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.700725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 472/946] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:20.702770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.807233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.811070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bf429-1535-4f4c-93f6-903be1112518 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.815144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:20.838535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf6626-0903-4b72-9498-81db03757acf None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:20.851366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.852362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.858667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.858667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.858667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.858667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:20.865568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:20.867608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:20.868627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:20.874970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 475/947] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:20.876319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:20.881418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:20.887590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.887590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.887590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.894817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:20.896782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-3245ab2d-9330-4dca-9c1d-7ca097cfcfb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:20.914454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:20.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:20.916403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:20.916403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:20.916403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:20.919151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 473/948] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:20.920582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b1f-5e24-4fc3-abeb-5dd561fa1f39 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:20.920776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b1f-5e24-4fc3-abeb-5dd561fa1f39 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:20.921053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752b1f-5e24-4fc3-abeb-5dd561fa1f39 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:20.926524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 474/949] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:20.926524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752b1f-5e24-4fc3-abeb-5dd561fa1f39 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.002188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072f4c0-f860-478f-866b-0097d48c7023 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.040988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.041605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.041891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.042018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752b1f-5e24-4fc3-abeb-5dd561fa1f39 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.042221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.042971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:21.043279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.044671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e493b-7644-4c65-aa1b-89516984b918 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.050657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 476/950] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:21.050986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.051170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.051454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.071036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:21.072157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:21.083835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.084561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.085558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.085839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.086491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:21.086825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.088202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.089711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.091450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.106294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 475/951] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:21.106294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.158648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 477/952] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:21.158648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.218154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533d114-421f-43f0-8f12-c469a135653c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.260126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.261035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.261323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.261597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.262234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:21.262535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.264397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.274851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 476/953] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:21.277879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.283915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ef199-cbe8-46ef-9ee2-9809af66e429 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.321221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.322332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.322872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.323443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.330978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.332814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:21.345145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:21.346114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:21.347098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:21.349625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 478/954] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:21.349737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.355671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.375330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.386038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ba49-0e6c-4068-ba7e-5eeac318c7cf None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.432145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.432775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.433062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.433384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.434469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.436058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.477095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf29a6b-024d-404e-8b0a-adfa86c19d41 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.516216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.516897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.517155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.517433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.518134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:21.518442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.519964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.532378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 479/955] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:21.534895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.542502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 477/956] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:21.543512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.622704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.657664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dc99-49da-46ad-9312-e00f1198996b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.697847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac9dde-896e-49b5-a13a-fca2d7086b5f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.701126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.701978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.702494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.702946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.703691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.705275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.710056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:21.714906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:21.716051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:21.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 480/957] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:21.723157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:21.751112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.752219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.752811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.753282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ecc0db48a09441eb68340c620de4263/roles/8788f0cdca9e477cbbcee427c9898dcd` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.757406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=4ecc0db48a09441eb68340c620de4263, role_id=8788f0cdca9e477cbbcee427c9898dcd)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.759218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.837944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b8db8-139b-43be-a957-9d5d6f934206 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:21.873073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:21.873606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:21.873993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:21.874271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:21.874935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:21.875293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:21.883459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac56ff-a446-45c3-9b17-6acb03130573 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:21.885992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:21.890899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 478/958] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:21 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/4ecc0db48a09441eb68340c620de4263/roles/8788f0cdca9e477cbbcee427c9898dcd => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:56:21.891631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 481/959] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:21.901579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:22.036159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1fef73-6969-4578-9aad-594375c00c0f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:22.075541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:22.076521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:22.077008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:22.077488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:22.078321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:22.078855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:22.080495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3767961-c8e4-4ec4-bd1e-d59b7a4deaf9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:22.086046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 479/960] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:22.373086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d86d1-e950-4354-ade6-d8e758bdffe8 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:22.373493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d86d1-e950-4354-ade6-d8e758bdffe8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:22.373671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d86d1-e950-4354-ade6-d8e758bdffe8 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:22.377181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 482/961] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:22.382701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:22.383263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:22.383760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:22.405100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:22.406746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:22.435815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:22.480931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 480/962] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:22.754726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d86d1-e950-4354-ade6-d8e758bdffe8 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:22.871339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d86d1-e950-4354-ade6-d8e758bdffe8 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:22.911532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:22.911532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:22.911532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:22.911532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:22.911957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:22.912213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:22.918641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING keystone.server.flask.common [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 05:56:22.971892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.sql.core [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 05:56:22.971892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 05:56:22.971892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [parameters: {'user_id': '2a2491aaab774a579adfeae46df8387a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 05:56:22.971892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81761) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application result.read() Aug 26 05:56:22.989229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:56:22.990193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application yield resource Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application next(self.gen) Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise self.value Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application session.commit() Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 05:56:22.992684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self.session.flush() Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application compat.raise_( Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 05:56:22.993615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 05:56:22.994497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application conn.query(q) Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application result.read() Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 05:56:22.995417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [parameters: {'user_id': '2a2491aaab774a579adfeae46df8387a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:22.997364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 05:56:23.002751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:23.003783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 483/963] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Aug 26 05:56:22 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:23.004200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:23.111576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39f1ca-5952-4951-9664-517fbb250d62 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:23.148975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:23.150212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.150719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.151174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.159375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:23.161105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-000cbeae-7bfd-47fc-a862-fbd0b33a5440 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:23.174318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:23.175312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 05:56:23.176331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:23.178532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 481/964] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:23.184891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:23.261123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:23.300896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da2409-7b9a-4dff-af9a-5247a86c86c7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:23.337822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:23.338435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.338748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.339027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.339788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:23.340093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:23.341727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfc39-cfa5-4fe6-99ab-96ead4f3a39c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:23.382530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 484/965] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Aug 26 05:56:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:23.562803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd710-daab-4fca-9ec1-7d8c70e35a80 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.563168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd710-daab-4fca-9ec1-7d8c70e35a80 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.563365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddd710-daab-4fca-9ec1-7d8c70e35a80 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.565010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 482/966] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:23.571257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695785a-1a38-4b74-b06c-933186d71973 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.571553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695785a-1a38-4b74-b06c-933186d71973 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.571836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695785a-1a38-4b74-b06c-933186d71973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.591723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5695785a-1a38-4b74-b06c-933186d71973 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:23.593246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5695785a-1a38-4b74-b06c-933186d71973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:23.665741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 485/967] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:23.907253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26cc-a01f-4752-853d-27e50d198d95 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.907546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26cc-a01f-4752-853d-27e50d198d95 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.907821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd26cc-a01f-4752-853d-27e50d198d95 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.909572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 483/968] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:23.914845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:23.915337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:23.915419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:23.934064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:23.935212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:23.953232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd26cc-a01f-4752-853d-27e50d198d95 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:23.982093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:23.993517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.022263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 486/969] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:24.069945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd26cc-a01f-4752-853d-27e50d198d95 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.106865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.107867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.108408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.108894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.109684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.111356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-4f6cdf95-38cf-40a4-a9ed-bf8cf4b9cb22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:24.121519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:24.122976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:24.123823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:24.124576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 484/970] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:24.129720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.136482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af007cf-2cdb-4b94-a925-144180f1f6b4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.137370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af007cf-2cdb-4b94-a925-144180f1f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.137678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af007cf-2cdb-4b94-a925-144180f1f6b4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.139211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 485/971] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:24.144120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.144407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.144676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.163830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:24.165038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:24.237982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf602d3b-8930-475e-8de1-8e05f69a09b0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.240497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 486/972] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:24.274169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.275433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.275767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.276056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.276714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:24.277055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.278459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.290442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 487/973] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:24.291880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.308074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.312715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.363223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.403491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c49472-5b46-4dbb-bb7c-68ddee81ac89 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.422496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35a5d9-cc45-4a53-8bf4-0a9afc9ba767 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.441604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.442212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.442489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.442795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.443429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:24.443727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.445681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.451267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 487/974] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:24.457170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.459213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.460060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.460535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.460981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.471985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:24.477463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:24.478407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:24.479268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 488/975] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:24.484417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.549112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.566302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece1cf7-cdd5-4b00-ab97-4daed2b1c792 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.585594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3755fdc-fbaf-4b8d-b7ef-2eea41968157 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.587934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.602098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.602720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.603005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.603277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.603989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] RBAC: Adding query filter params (interface=admin, service_id=8973a0dfa13f49b496fd1d29d7cbd7f4, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:24.604296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.605673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.610910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 488/976] 10.222.0.40 () {56 vars in 1249 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/endpoints?service_id=8973a0dfa13f49b496fd1d29d7cbd7f4&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:24.611680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.620930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.621547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.621835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.622115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.622934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:24.623248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.624623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.632937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 489/977] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:24.633754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.714443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb06e2-755d-49ee-a6c4-c461b7c402dc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.737477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c60dc9-ff7e-4f27-a1fa-843492db9fb1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.749874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.750489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.750821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.751097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.751758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.753140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.771676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.778549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.778549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.778549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.778549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.778549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:24.781514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 490/978] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 05:56:24.782367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:24.817720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:24.861469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 489/979] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:24 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:24.909998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc4713-561f-4b2d-b3d8-fe993d33e2e3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:24.967429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:24.968101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:24.968355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:24.968631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:24.981317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:24.982868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:25.000511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:25.001533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:25.003153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:25.003153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:25.003153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:25.003153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 491/980] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:25.008208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:25.017616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:25.121448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463194b-b39a-4667-ae5c-ad3f904f4a98 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:25.158472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:25.159141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:25.159394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:25.159666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:25.160318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:25.160624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:25.162008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:25.175164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 490/981] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:25.181442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:25.268032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:25.290935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6954b91-e63d-4626-b63a-c4b0e1e23dc4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:25.330130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:25.330823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:25.331113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:25.331395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:25.340310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:25.341779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:25.354168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:25.355689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:25.356751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 492/982] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:56:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:25.363463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:25.441090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:25.480562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddb1c-02db-46f3-9223-f3de34205155 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:25.517811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:25.518409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:25.518709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:25.518978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:25.519698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:25.519998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:25.521439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:25.543933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 491/983] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Aug 26 05:56:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:25.550009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:25.671468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe156a-05a4-4471-9f02-eeefc1a7d2ab None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:25.711490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:25.712016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:25.712293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:25.712560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:25.721538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:25.787542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:25.789767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:25.902699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 493/984] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:25 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:25.908714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:26.024693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a20f1f-3405-4d03-89d2-8f222040135a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:26.048813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:26.059936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:26.060568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.060841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.061115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.071163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:26.072743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:26.087944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:26.088958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:26.089877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:26.089877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:26.089877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:26.089877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 492/985] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:26.095937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:26.136258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:26.199050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2541c-9f9e-4733-9710-9a9af4982d1a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:26.234385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:26.235032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.235316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.235592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.236237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:26.236541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:26.237918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:26.254518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 494/986] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:26.255702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:26.366531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aee9b-658c-46f1-9a5b-ab65cebb50fb None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:26.408547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:26.409387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.409679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.409951 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.417391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:26.418877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:26.430598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:26.431571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:26.432497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 493/987] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:26.438915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:26.550321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:26.555720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969ee0d-d550-49fc-b996-6a3f2f7bb7e3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:26.592713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:26.593697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.594183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.594685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.595589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:26.596138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:26.597765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7b277-99e1-4225-abe9-5d2de62983ae None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:26.615545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 495/988] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:26.617786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:26.662724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:26.736673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f74357-4c91-4978-b3c4-9fca4f742ca9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:26.760531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a15eb-756f-4f79-9e9d-49659694a234 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.760821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a15eb-756f-4f79-9e9d-49659694a234 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.761099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a15eb-756f-4f79-9e9d-49659694a234 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.762579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 496/989] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:26.767915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.768201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.768481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.777058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:26.777909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.778400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.778881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.779782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] RBAC: Adding query filter params (role.id=8788f0cdca9e477cbbcee427c9898dcd, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=4ecc0db48a09441eb68340c620de4263) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:26.780262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:26.781858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a073e3-23e8-453a-a337-5e795eea8292 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:26.788428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 494/990] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:26 2022] GET /identity/v3/role_assignments?user.id=4ecc0db48a09441eb68340c620de4263&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=8788f0cdca9e477cbbcee427c9898dcd => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:26.789152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:26.790413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:26.865404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936c0f-a196-48c5-a3ee-1b4866d1b011 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.866220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936c0f-a196-48c5-a3ee-1b4866d1b011 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.866532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06936c0f-a196-48c5-a3ee-1b4866d1b011 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.868123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 495/991] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:26.883931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:26.884239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:26.884496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:26.885117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 497/992] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:26.910680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:26.910680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:26.992371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 496/993] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:26 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:27.158356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.258646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.261614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.274735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4436d-52cb-4652-8f7b-c42449d5ec28 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.300837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.315363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.316008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.316289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.316565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.317155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.318652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:27.328221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:27.329265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 05:56:27.330149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:27.331278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 498/994] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:27.337125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.395209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8cf8-6f6e-4054-b2ee-c7e29442ff95 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.447128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.447689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.447971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.448249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.448838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.450213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.452387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4d3c-0ab6-4a68-ad5f-e7dced1cff55 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:27.460046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:27.463212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:27.464354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 497/995] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:27.466902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.495308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.495996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.496213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.496487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.497127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:27.497449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.498842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.504165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 499/996] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:27.510385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.530610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.578358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.586179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1de23-db7b-447d-b0c4-c23140a952d8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.624703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8d8a4-7a1e-4a27-a10d-d91f4f44c38a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.629576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.630200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.630484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.630793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.631463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:27.631769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.633351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.643286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 498/997] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:27.644315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.666916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.667546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.667832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.668109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.668764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:27.669069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.670643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.681100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 500/998] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:27.682317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.766040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d94c-12d2-4758-b029-0fdebdc9338b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.789195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.807653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.811975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.811975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.811975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.811975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:27.811975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.812917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.816233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c63e9-ee45-4b06-a472-897025545b71 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.819017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 499/999] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:27.824604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:27.858821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:27.870435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:27.871205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:27.871488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:27.871761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:27.872426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:27.873901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:27.943320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c298012-7d60-4aad-9e94-661e65b28fc1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:27.982644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 501/1000] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:27 2022] POST /identity/v3/endpoints => generated 325 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:28.000815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:28.001460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:28.001741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:28.002019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:28.002801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] RBAC: Adding query filter params (interface=public, service_id=5caad5dd6225414da6e5d88678d1964e, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:28.003117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:28.004546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2d7da-5bf7-493e-b72c-ce6d200e5bd5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:28.011688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 500/1001] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:27 2022] GET /identity/v3/endpoints?service_id=5caad5dd6225414da6e5d88678d1964e&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:28.017586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:28.134449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b0e9f-9855-4246-8764-10f6818c3a3a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:28.176379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:28.178964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:28.179350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:28.179522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:28.180159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:28.183956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5ddb8-0509-4c3e-b8fe-728108824bf5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:28.191384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 502/1002] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:28 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 05:56:28.803553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f92bf-d029-4bb7-b467-cc74d9a43c89 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:28.804236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f92bf-d029-4bb7-b467-cc74d9a43c89 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:28.804236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f92bf-d029-4bb7-b467-cc74d9a43c89 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:28.805521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 501/1003] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:28.811201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:28.811556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:28.811905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:28.820422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:28.831888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:28.835211 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:28.908105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 503/1004] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:29.179226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f92bf-d029-4bb7-b467-cc74d9a43c89 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:29.266035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:29.285861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f92bf-d029-4bb7-b467-cc74d9a43c89 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:29.323457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:29.324095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:29.324399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:29.324676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:29.325261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:29.326655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:29.336086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:29.337155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:29.337999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:29.342132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 502/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 05:56:29 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:29.343418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:29.447567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27293c-f620-4fa6-b6c6-d001ccf8e71b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:29.483008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:29.483958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:29.484427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:29.484887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:29.485702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] RBAC: Adding query filter params (name=image) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:29.486192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:29.487788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0918c7a4-4a1a-4d66-97fd-09e5bfb48dbb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:29.489021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:29.498678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 504/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:29.501065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:29.607984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a0acc-c989-4f8e-a273-4cc78ad480e7 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:29.643414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:29.644364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:29.644830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:29.645287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:29.646112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] RBAC: Adding query filter params (type=image) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:29.646600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:29.648858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5143d-dbd3-4c03-9b4b-dc800f187e81 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:29.662082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 503/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:29.948070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee22c-0bea-4454-8eec-dc64892b9644 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:29.948392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee22c-0bea-4454-8eec-dc64892b9644 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:29.948657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee22c-0bea-4454-8eec-dc64892b9644 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:29.949959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 505/1008] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:29.955497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c945-57be-43ec-8fe7-2abc35b517a6 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:29.955954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c945-57be-43ec-8fe7-2abc35b517a6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:29.956029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8c945-57be-43ec-8fe7-2abc35b517a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:29.975380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-40c8c945-57be-43ec-8fe7-2abc35b517a6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:29.976568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-40c8c945-57be-43ec-8fe7-2abc35b517a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:29.980601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:30.046861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 504/1009] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:29 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:30.159065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088e7c-ee8d-403d-9831-124f0c461382 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.159367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088e7c-ee8d-403d-9831-124f0c461382 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.159629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88088e7c-ee8d-403d-9831-124f0c461382 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.161110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 506/1010] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:30.167031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.167316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.167575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.186512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:30.187727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:30.259641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 505/1011] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:30.322580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088e7c-ee8d-403d-9831-124f0c461382 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:30.426758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88088e7c-ee8d-403d-9831-124f0c461382 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:30.461909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:30.462529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.462841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.463120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.464233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:30.465664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:30.467592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 507/1012] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:30.473290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:30.532680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:30.554238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:30.588398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c978a2-c546-493b-a8d7-f4719e609b11 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:30.605072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:30.626657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:30.626657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.626928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.627169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.627765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:30.629259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:30.641064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fd634-7791-4b60-a4ea-1895d6125ce2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:30.675927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:30.676562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.676894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.677115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.677695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:30.679207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:30.687676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:30.692971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:30.694227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 508/1013] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:30.695797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:30.718974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 506/1014] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:30 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:30.814160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e551b6f-8926-49ce-8075-194891b9c31e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:30.845643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:30.853142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:30.853800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:30.854080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:30.854355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:30.855053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:30.855383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:30.856776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:30.862347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 509/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:30.869201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:31.042535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a63795-12dd-493c-bd63-171cde03c399 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:31.065228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:31.077809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:31.078434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:31.078748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:31.079025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:31.079670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:31.079975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:31.081374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becdf99-26dc-4eab-9885-5748ec16408a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:31.087339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 507/1016] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Aug 26 05:56:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:31.093158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:31.198834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a09f-1726-44ca-b8ab-0ebe3a0c3c64 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:31.244770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:31.244770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:31.244770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:31.244770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:31.245411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:31.246755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a971cf-044d-4374-bea6-902aa5cfec37 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:31.387177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 510/1017] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:31.618392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817e21-fdba-4582-8ad1-934da6340f5c None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:31.618948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817e21-fdba-4582-8ad1-934da6340f5c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:31.619248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817e21-fdba-4582-8ad1-934da6340f5c None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:31.621341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 508/1018] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:31.628399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f054-d9e6-4368-a301-a9fdc2664084 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:31.628912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f054-d9e6-4368-a301-a9fdc2664084 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:31.629007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f054-d9e6-4368-a301-a9fdc2664084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:31.649045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e813f054-d9e6-4368-a301-a9fdc2664084 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:31.650435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e813f054-d9e6-4368-a301-a9fdc2664084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:31.702551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:31.731246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 511/1019] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:31 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:32.009963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60817e21-fdba-4582-8ad1-934da6340f5c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:32.118693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60817e21-fdba-4582-8ad1-934da6340f5c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:32.153337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:32.153948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:32.154230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:32.154505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:32.155191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:32.156558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f7b5b-9f08-4799-8ff4-6d34ebb870be None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:32.262754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 509/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:32 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:32.742357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed9c2-9515-4295-86ea-5daefd9b7dbc None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:32.742905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed9c2-9515-4295-86ea-5daefd9b7dbc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:32.742986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eed9c2-9515-4295-86ea-5daefd9b7dbc None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:32.744432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 512/1021] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:32.749733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:32.750099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:32.750448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:32.770001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:32.771242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:32.819448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:32.849184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 510/1022] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:33.125644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed9c2-9515-4295-86ea-5daefd9b7dbc None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:33.234590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eed9c2-9515-4295-86ea-5daefd9b7dbc None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:33.272625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:33.273358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:33.273619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:33.273898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:33.275352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:33.276827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f61e9-755d-4520-ba82-10a0e76dbf20 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:33.278702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 513/1023] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:33.284305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:33.400039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a846121-0ba5-4c65-9b7f-754c442ce198 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:33.441489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:33.442112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:33.442395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:33.442697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:33.443368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:33.445235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:33.469967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ddd4-b18a-4e62-b5a6-5fe3093ad98c None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:33.470263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ddd4-b18a-4e62-b5a6-5fe3093ad98c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:33.470538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ddd4-b18a-4e62-b5a6-5fe3093ad98c None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:33.471914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 514/1024] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:33.477418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:33.477708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:33.477990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:33.496601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:33.497799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:33.538847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:33.566517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 515/1025] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:33.650753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 511/1026] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:56:33 2022] POST /identity/v3/users => generated 285 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:56:33.841557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:33.962045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f83e5-74ac-46ba-a7bb-2f932b27eab7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:33.999020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:33.999660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:33.999982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.000265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.011409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:34.016188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:34.017142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:34.017980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 516/1027] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:34.023693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.054302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.130417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd8e97-d2d9-4925-8788-9ed0c4435d0d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.171209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.172141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.172584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.173003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.173817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:34.174270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.175818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56798596-977c-4000-b35b-115f9b1859d8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.185355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 512/1028] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:34.190675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.205639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d1d1-89ed-40a9-a9cb-29c139cd65af None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.205925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d1d1-89ed-40a9-a9cb-29c139cd65af None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.206200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87d1d1-89ed-40a9-a9cb-29c139cd65af None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.207570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 513/1029] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:34.213272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.213560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.213897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.231835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.232406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:34.233593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:34.304410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d28f0-fdf6-4f57-8ad8-b7b9746dd044 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.330698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 514/1030] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:34.345866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.346661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.347029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.347377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.358602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.360263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:34.377373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:34.378383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:34.379307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:34.379307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:34.379307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:34.382975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 517/1031] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:34.385054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.408312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.496175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735eac-8f90-4b86-af81-d2b940adab5b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.531691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.532301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.532579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.532849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.533486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:34.533786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.535199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.551559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 515/1032] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:34.552917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.624255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.655223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.671179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7fea9-977a-42ab-afa8-b9c87b3acf4d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.728453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.729410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.729744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.729857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.730263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.741176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.742810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.752785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be426597-3f62-43bf-9a48-f561535a81f8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:34.755672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:34.757139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:34.758100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 518/1033] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:34.763299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.800526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.801152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.801427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.801697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] PATH_INFO: `/v3/services/image` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.802267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.804529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:34.813348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:34.814365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:34.820627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 516/1034] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:34.822986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.902585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c51bc-be56-4c9d-b448-57851dec64b6 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.946186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.946827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.947111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.947382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.948117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:34.948426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.948731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831582f7-37e3-47be-809d-e59383ff1acd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:34.949853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:34.969150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 519/1035] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:34.974388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:34.978739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:34.992468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:34.993089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:34.993397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:34.993645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:34.994282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] RBAC: Adding query filter params (name=image) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:34.994583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:34.995999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.001151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 517/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:35.007486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.018695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.094116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdb18b-7d49-47ad-bea6-a12fa7d4487f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.119466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1b17-2cb1-4ce6-8ec9-7b6ed9a93069 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.130480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.131193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.131480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.131819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.156396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.157004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.157294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.157553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.158226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] RBAC: Adding query filter params (type=image) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:35.158483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.159900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.165220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 518/1037] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:35.170967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.206446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.207956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.211433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.247127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.277979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a5a81-bc81-4ddd-a492-895f0a5dd089 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.313487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.314202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.314487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.314793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.315502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] RBAC: Adding query filter params (interface=public, service_id=715ee169c4624082a9f42a800bcce077, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:35.315812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.317188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.322756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 520/1038] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:34 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:35.326658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 519/1039] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/endpoints?service_id=715ee169c4624082a9f42a800bcce077&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:35.328588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.329884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.447034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a944b-6aa7-48d6-b3bb-19e27c9cd248 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.449464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311251d2-4a1f-4cf9-aa5b-e812deb192dc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.451444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.486973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.487562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.487684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.487903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.488181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.488278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.488555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.488787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.489417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.490942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.493381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.499794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 520/1040] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 05:56:35.513873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.515478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-f7ff7f65-eeca-490a-aea6-4fe54b241e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:35.534057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:35.535114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:35.536397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:35.536397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:35.536397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:35.539106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 521/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:35.541399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.663423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8341c4c-b9cb-4fcd-b86e-63a95d1d6269 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.699061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbda18-ad26-4eaa-8836-03885083bdab None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.699375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbda18-ad26-4eaa-8836-03885083bdab None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.699620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbda18-ad26-4eaa-8836-03885083bdab None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.700468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.701151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 522/1042] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:35.701277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.701365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.701607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.702758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:35.702758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.704233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.706774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.707065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.707351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.719923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 521/1043] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:35.726367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:35.727922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:35.729166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:35.798739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:35.808067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 523/1044] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:35.845532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d06c-7b41-4726-bce2-13487c86c553 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:35.880459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:35.881185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:35.881465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:35.881818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:35.888424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:35.889855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:35.905322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:35.906264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:35.915521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 522/1045] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:35.918004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.008085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:36.023204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a1416-83d3-49e7-98b5-14449082f9f3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.058352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.058933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.059222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.059503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.060198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:36.060507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:36.061988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:36.073979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 524/1046] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:36.079887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.083306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.191524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97312c4b-d539-48c8-920a-5792a67ed30d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.196945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9b95d-0350-46da-be9f-c100d36df186 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.215973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:36.233828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.234502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.234876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.235130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.235840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] RBAC: Adding query filter params (role.id=8788f0cdca9e477cbbcee427c9898dcd, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:36.236171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:36.237597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:36.238130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.238809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.239097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.239939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.245308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 523/1047] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:36 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=8788f0cdca9e477cbbcee427c9898dcd => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:36.253706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:36.259685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:36.260737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:36.261574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:36.266247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 525/1048] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:36.267443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.393659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d88e-217e-4319-b51f-1a096b93d080 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.435244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.435861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.436145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.436427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.437097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:36.437403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:36.438801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:36.448525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 524/1049] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:36.455440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.469707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:36.605178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa59f31-84cf-44b8-8b37-cc52004ebfb7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.641467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.642110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.642393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.642698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] PATH_INFO: `/v3/users/a05c90b11f1045b798449e8c2615b507` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.658249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c90b11f1045b798449e8c2615b507)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:36.659738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:36.666720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 526/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:36 2022] GET /identity/v3/users/a05c90b11f1045b798449e8c2615b507 => generated 285 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:36.668443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:36.722835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:36.780074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743e9e4-4300-4f6a-8515-aa5a7cba35ee None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:36.824159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:36.824792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:36.825088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:36.825348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:36.905839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:36.907352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:36.959969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:37.016668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 525/1051] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:36 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:37.022970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:37.090878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:37.134793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a141dd-64a4-4613-85ec-7553efc9e000 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:37.175308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:37.176388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:37.176928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:37.177420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:37.179235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:37.180990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf824b-0038-4eca-ad73-830ac84ad35f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:37.189436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:37.191993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 527/1052] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:37 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:37.310434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c5e1-5314-4ea2-9a24-7a35e9ea1749 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:37.353972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6, user.id=a05c90b11f1045b798449e8c2615b507) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:37.358643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a6a3b-1b2a-4881-ae93-08d49a4ad453 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:37.364675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 526/1053] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:37 2022] GET /identity/v3/role_assignments?user.id=a05c90b11f1045b798449e8c2615b507&scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:37.572611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ea1b-25d5-48a7-bf97-311e08268934 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:37.572811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ea1b-25d5-48a7-bf97-311e08268934 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:37.573080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489ea1b-25d5-48a7-bf97-311e08268934 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:37.574729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 528/1054] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:37.583159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2327-60ce-42e9-8c27-be1c9e85fc22 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:37.583448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2327-60ce-42e9-8c27-be1c9e85fc22 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:37.583794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2327-60ce-42e9-8c27-be1c9e85fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:37.606437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7a6f2327-60ce-42e9-8c27-be1c9e85fc22 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:37.608157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7a6f2327-60ce-42e9-8c27-be1c9e85fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:37.613350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:37.718691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 527/1055] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:38.002532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ea1b-25d5-48a7-bf97-311e08268934 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.172761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489ea1b-25d5-48a7-bf97-311e08268934 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:38.206570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:38.213000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:38.213721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.213881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.214156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] PATH_INFO: `/v3/services/image` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.214796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:38.225560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:38.225706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a56da7-1de2-45cb-a145-ef2d7f819609 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.225706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a56da7-1de2-45cb-a145-ef2d7f819609 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.225706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a56da7-1de2-45cb-a145-ef2d7f819609 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.225706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 528/1056] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:38.225964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:38.226951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 05:56:38.231825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:38.232691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.232691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.232691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.232897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 529/1057] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Aug 26 05:56:38 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:38.233672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.248493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:38.249822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:38.322510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 529/1058] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:38.352357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb074103-f26f-4ad7-8dc1-feddc7a5cd16 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:38.389857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:38.390470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.390873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.391167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.391829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] RBAC: Adding query filter params (name=image) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:38.392131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:38.393501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:38.404518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 530/1059] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:38.405495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.483915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:38.521020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc076f-fe3e-4e35-a1a6-8bf92e1bd608 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:38.558675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:38.559526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.559950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.560405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.561202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:38.561648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:38.563186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:38.572832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 530/1060] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:38.575771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.590533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.660664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:38.717193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e827b-9730-42f8-947f-0a960707ce43 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:38.731839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd6169-a63d-4277-a4bd-6eb8840792d8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:38.764708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:38.765243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.765528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.765806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.766466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:38.767882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:38.786525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:38.787854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:38.788711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:38.788975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:38.796037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:38.800009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:38.808071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:38.809210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:38.810073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 531/1061] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:38.815654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:38.866656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 531/1062] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:56:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:38.965530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8af4-5fb9-436d-a6b8-9b28aaa4ae64 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.014058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.014800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.015020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.015352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.016018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:39.016374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.017817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.028094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 532/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:39.033970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.153490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2e3f3-6f25-4ade-a996-cfcaa8f046ed None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.170499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:39.190428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.196716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.196716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.196716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.203674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.205163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c782e513-106f-4131-8269-2231c635d5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:39.221130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:39.223506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:39.225418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:39.225418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:39.225418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:39.225418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 532/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:39.231930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.303777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:39.356214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f17e-b565-4e81-ad7a-a5bd4c34c6d1 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.356562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f17e-b565-4e81-ad7a-a5bd4c34c6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.356787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f17e-b565-4e81-ad7a-a5bd4c34c6d1 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.358166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 533/1065] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 05:56:39.363233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.363606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.363792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.370587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059ce1-2128-447f-bfe6-78a6b9436a10 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.385786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:39.386816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:39.407085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.407857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.408256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.408632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.409391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:39.409702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.411115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.429211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 533/1066] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:39.430384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.469004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 534/1067] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:39.543790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383410b1-3c19-421a-9292-6c21149abc0c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.558576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:39.581003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.581643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.581967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.582244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.591277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.592736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:39.604755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:39.605722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:39.606750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 534/1068] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:39.611714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.712435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:39.716987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec18bff-fda5-403d-9e00-e17cc881b08d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.741437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.751771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.752390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.752689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.752962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.753704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:39.754024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.755496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.776218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 535/1069] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:39.781889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.854814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655e76e-1092-410e-a6ee-f886d223ecee None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.906760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.907404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.907689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.907962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.919958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722e43d-7181-466d-8285-7cdd9aa68ebd None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:39.923889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:39.930437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:39.931444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:39.932322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:39.933057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 535/1070] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:39.937943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:39.963409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:39.964057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:39.964340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:39.964615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:39.980415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:39.980655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:39.982046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:39.998236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:39.999528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:40.001250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:40.001250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:40.001250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:40.001250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 536/1071] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:40.005264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.020385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:40.063537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a715e-ab4a-43ef-ac71-0cfc728961c8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.104882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.105400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.105684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.105956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.106611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:40.106946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.108314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.118653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 536/1072] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:40.124294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.124982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d642f-ab9e-4c76-95bb-d6be5d91363b None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.164016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.164985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.165485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.165967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.166976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:40.167460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.169182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.184895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 537/1073] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:40.190569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.234307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:40.241721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bb74-509a-4374-830a-39b43f86db66 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.272905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:40.281368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.282010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.282606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.282606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] PATH_INFO: `/v3/users/a05c90b11f1045b798449e8c2615b507` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.293937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c90b11f1045b798449e8c2615b507)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.293937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.293937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 537/1074] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:40 2022] GET /identity/v3/users/a05c90b11f1045b798449e8c2615b507 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:40.294250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.324598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dead47-5d0f-41a1-9e6e-e743e4c98e54 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.365570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.366492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.367112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.367900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.375601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.377051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:40.391199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:40.393003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:40.393966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 538/1075] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:40.399438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.427117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959fa59-06cd-4b92-8c32-ccd9d5000c47 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.471288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.471936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.472344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.472796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.474080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.476384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.478039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 538/1076] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:40 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:40.484815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.522144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:40.525092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a8105-4ff5-4f2c-94f1-c3b0dfea873d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.557947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:40.566789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.567417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.567704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.567984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.569258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:40.569563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.571051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.583746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 539/1077] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:40.590756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:40.617880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ee203-293b-4b17-a470-9d34aadc3438 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.667933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.668597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.668877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.669149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6/users/a05c90b11f1045b798449e8c2615b507/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.671094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d003e81fbd24fa9b283cd4c8535a1b6, user_id=a05c90b11f1045b798449e8c2615b507, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.672772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce516e-bf64-4435-b288-1a4e437a6480 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.701072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e95a-81a7-49be-9294-261e58b702cb None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:40.737071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:40.737790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.738080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.738360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/8788f0cdca9e477cbbcee427c9898dcd` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.741195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=9dd40d8b3cb543feab8424f1d87b7126, role_id=8788f0cdca9e477cbbcee427c9898dcd)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:40.743344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0913ec9-ffc5-44bf-8e7a-8ecb28074b1f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:40.747182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 539/1078] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:40 2022] PUT /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6/users/a05c90b11f1045b798449e8c2615b507/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:56:40.827033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 540/1079] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:40 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/9dd40d8b3cb543feab8424f1d87b7126/roles/8788f0cdca9e477cbbcee427c9898dcd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 05:56:40.931616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a161-68b5-4118-be47-165a06f23d36 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.931959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a161-68b5-4118-be47-165a06f23d36 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.932154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a161-68b5-4118-be47-165a06f23d36 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.933733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 540/1080] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:40.939937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:40.940238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:40.940499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:40.961136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:40.962368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:41.017305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:41.042993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 541/1081] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:41.326226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a161-68b5-4118-be47-165a06f23d36 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:41.419190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:41.430024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a161-68b5-4118-be47-165a06f23d36 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:41.464275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:41.464878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:41.465156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:41.465427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:41.532722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:41.534166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:41.620365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:41.631623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 541/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:41.637192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:41.738045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a9d86-435e-48df-8e5e-04a103efce69 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:41.772951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:41.773583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:41.773877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:41.774131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:41.784656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:41.786100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-d77125c4-f21b-4837-b43a-8635e828a888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:41.802361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:41.803605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:41.827230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:41.827230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:41.827230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:41.827230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 542/1083] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:41.827493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:41.922047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9782a-b54f-474e-b5a6-596d8d241210 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:41.960118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:41.961093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:41.961569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:41.962054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:41.962920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:41.963431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:41.964980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8707918-f50f-4f80-87fd-4f7d6e5631d9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:41.977737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 542/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:42.705892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80fe5d-60be-4d67-ad21-392069a74bbb None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:42.706454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80fe5d-60be-4d67-ad21-392069a74bbb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:42.706454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80fe5d-60be-4d67-ad21-392069a74bbb None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:42.707991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 543/1085] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:42.713852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574457-64ad-4507-98b7-578790988d23 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:42.714218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574457-64ad-4507-98b7-578790988d23 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:42.714566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574457-64ad-4507-98b7-578790988d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:42.732845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-63574457-64ad-4507-98b7-578790988d23 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:42.734046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-63574457-64ad-4507-98b7-578790988d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:42.781535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:42.785957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6c55d-49ea-4b67-a7de-e5b7cf123a52 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:42.786246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6c55d-49ea-4b67-a7de-e5b7cf123a52 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:42.786523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6c55d-49ea-4b67-a7de-e5b7cf123a52 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:42.787922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 544/1086] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:42.792935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:42.796434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:42.796434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:42.802588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 543/1087] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:42.811599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:42.812796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:42.838454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:42.882493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 545/1088] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:43.077707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574457-64ad-4507-98b7-578790988d23 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.156983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.179489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574457-64ad-4507-98b7-578790988d23 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.214427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.215090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.215369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.215640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.226115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:43.230937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:43.231917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:43.232776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 544/1089] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:43.238045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.263271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476640d5-357c-4840-a92b-1a9a5715072c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.286700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.298766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.299404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.299680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.299954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.311547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:56:43.316222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:43.317331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:43.318566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 546/1090] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:43.323675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.345969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.346799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c53b42-fb22-4ea7-84c0-e31a4871a998 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.383555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.384178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.384457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.384732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.385403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:43.385706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.387127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.395775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 545/1091] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:43.401956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.434152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03eb42-6cb9-45b6-a83f-490d5d4b9862 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.475679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c29af-842d-4371-88af-f035f796a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.476292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c29af-842d-4371-88af-f035f796a598 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.476577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c29af-842d-4371-88af-f035f796a598 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.476857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c29af-842d-4371-88af-f035f796a598 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.477532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c29af-842d-4371-88af-f035f796a598 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:43.477844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c29af-842d-4371-88af-f035f796a598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.479276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c29af-842d-4371-88af-f035f796a598 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.488037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 547/1092] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:43.494227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c29af-842d-4371-88af-f035f796a598 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.504194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076253f4-1293-4aa1-8580-883904604f7a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.505775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.542142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] PATH_INFO: `/v3/users/a05c90b11f1045b798449e8c2615b507` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c90b11f1045b798449e8c2615b507)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.548585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 546/1093] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/users/a05c90b11f1045b798449e8c2615b507 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:43.554925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.589805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.611063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c29af-842d-4371-88af-f035f796a598 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.650971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.651883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.652166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.657081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.659736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1b7e7-46cc-4d32-abf5-00388d4654a3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.664888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.666378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:43.684665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:43.693981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:43.695169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:43.695169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:43.695169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:43.695169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 548/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:43.696595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.710160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.711233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.711731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.712216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.760142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.791138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.792896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.810310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef87a11-d3db-48e0-aff7-d73520f777a0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.846261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:43.846901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:43.847184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:43.847461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:43.848113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:43.848421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:43.849835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:43.851437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:43.864784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 549/1095] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:43.868886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.902409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 547/1096] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 05:56:43.909518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:43.985233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85168af-25c4-48eb-aaab-0b97299fb5ff None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:43.986069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:44.019109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b52cea-71d6-4891-99f7-a84539aa0c0e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.024467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.025109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.025395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.025668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.033321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.034808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:56:44.048948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:44.050050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:56:44.051474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 550/1097] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:44.056030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.056926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.057566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.057842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.058111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.059258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.060683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.062352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 548/1098] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:43 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:44.069338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.117371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:44.165737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1be8b-0d32-4249-93db-11d2779e503d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.176577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbed50-8cbd-490f-af6d-03dff99096a1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.202177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.202595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.202908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.203185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.203920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:44.204228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.205670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.218323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.218959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.219244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.219516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.220237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6, user.id=a05c90b11f1045b798449e8c2615b507) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:44.220545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.221972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.223512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 551/1099] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 05:56:44.228112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 549/1100] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/role_assignments?user.id=a05c90b11f1045b798449e8c2615b507&scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:44.230028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.341338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab516a1e-3602-4017-a03d-e565762a31ec None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.358231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:44.380403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.381061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.381375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.381686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.453334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.454837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.564034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 552/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 05:56:44.565808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.569257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:44.673742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5276505-189c-4a66-a771-25173c14e72b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.707827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.708457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.709373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.709674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.720409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.721854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:56:44.736709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:44.737863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:56:44.738778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:56:44.738778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:56:44.738778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:44.738778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 550/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:56:44.743856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.758445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:44.854318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c39fb-087a-448d-b279-c562a171c135 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:44.891779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:44.892399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:44.892681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:44.892954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:44.893615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:44.893906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:44.895322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:44.913710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 553/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:44.914707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:44.970108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:45.017663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d9858-bb6f-42f4-884b-a5c46dd68b3b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:45.051187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:45.051810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:45.052090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:45.052390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:45.059101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:45.060557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:56:45.073126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:56:45.074146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:56:45.075238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 551/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:56:45.081869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:45.158997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:45.190474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebd994-87b2-4a74-98e3-0ee94d4de21e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:45.228148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:45.229119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:45.229634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:45.230223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:45.231600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:45.231907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:45.233364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b4d4d-ad97-4bdb-8688-9850e64c4850 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:45.246420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 554/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:45.253411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:45.359682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe942ef2-05e5-44df-8163-7f775fb853fa None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:45.373769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:45.394760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:45.395369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:45.395652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:45.395926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:45.396873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] RBAC: Adding query filter params (role.id=8788f0cdca9e477cbbcee427c9898dcd, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=9dd40d8b3cb543feab8424f1d87b7126) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:45.397111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:45.398594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5811-658d-44e1-873e-1ddcbf706418 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:45.405380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 552/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:45 2022] GET /identity/v3/role_assignments?user.id=9dd40d8b3cb543feab8424f1d87b7126&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=8788f0cdca9e477cbbcee427c9898dcd => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:46.177097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a531e-3b6a-46df-b01a-c66b8ca6f891 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:46.178278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a531e-3b6a-46df-b01a-c66b8ca6f891 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:46.178381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a531e-3b6a-46df-b01a-c66b8ca6f891 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:46.179698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 555/1107] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:46.185229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:46.185545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:46.185823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:46.204826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:46.205996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:46.276212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 553/1108] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:46.544035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a531e-3b6a-46df-b01a-c66b8ca6f891 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:46.651504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a531e-3b6a-46df-b01a-c66b8ca6f891 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:46.680621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:46.685210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:46.685832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:46.686117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:46.686390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:46.687524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:46.688956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad0209-e5b4-41c8-9c09-8a8410e4ca00 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:46.690514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 556/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:46.697243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:46.792735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78f1eb-47d7-4ee3-9bc3-e0abbff0020d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:46.802078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:46.828133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:46.828945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:46.829524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:46.829943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:46.830814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:46.832340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13914e-4b97-47aa-b165-2e2e87a384be None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:46.937465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 554/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:56:46 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:56:48.941406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684776-36ac-4b9d-ba58-c7ddae828615 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:48.941958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684776-36ac-4b9d-ba58-c7ddae828615 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:48.941958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98684776-36ac-4b9d-ba58-c7ddae828615 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:48.944262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 557/1111] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:48.949994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:48.950525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:48.950525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:48.970404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:48.971644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:49.045378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 555/1112] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:49.326757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98684776-36ac-4b9d-ba58-c7ddae828615 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:49.432874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98684776-36ac-4b9d-ba58-c7ddae828615 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:49.466719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:49.467375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:49.467662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:49.467943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] PATH_INFO: `/v3/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:49.469382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] RBAC: Authorizing `identity:get_role(role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:49.470825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:49.472200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 558/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:49 2022] GET /identity/v3/roles/c17cdb7e94834eee81063576196e3fd9 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:49.478858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:49.573269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:49.579937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3351dd-ef37-48c0-82f9-a086848dd62e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:49.614561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:49.615886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:49.616167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:49.616445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] PATH_INFO: `/v3/users/0263124817e94b149f7b8ddf3399307a` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:49.631174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] RBAC: Authorizing `identity:get_user(user_id=0263124817e94b149f7b8ddf3399307a)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:49.642700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:49.642700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 556/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:49 2022] GET /identity/v3/users/0263124817e94b149f7b8ddf3399307a => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:49.642905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:49.709562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:49.743905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6a6de-63f7-4389-95e0-b8b049300e8a None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:49.779706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:49.780319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:49.780601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:49.780879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:49.852038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:49.853451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:49.923361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:50.023746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 559/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:50.030182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:50.216703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2344ff2-8f57-4654-a390-94fd71d86823 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:50.251954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:50.252593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:50.252874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:50.253146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:50.254296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:50.256462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec567d-1c0b-478d-a005-c7252ab01db9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:50.258155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 557/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:50 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:50.264922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:50.376409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badac39-cea0-4a22-86e6-7f7cf5d92cae None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:50.412890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:50.413539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:50.413819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:50.414094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:50.414850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6, user.id=0263124817e94b149f7b8ddf3399307a) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:50.415161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:50.416594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e08590-21f2-44b4-bc82-7d9e3468f6c7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:50.422020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 560/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:50 2022] GET /identity/v3/role_assignments?user.id=0263124817e94b149f7b8ddf3399307a&scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:52.358268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbef2d-e3da-4175-b7c0-8da3d8be3ccc None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:52.358864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbef2d-e3da-4175-b7c0-8da3d8be3ccc None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:52.358864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbef2d-e3da-4175-b7c0-8da3d8be3ccc None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:52.360196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 558/1118] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:52.365746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:52.366266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:52.366433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:52.384798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:52.385979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:52.412896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:52.455037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 561/1119] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:56:52.725636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbef2d-e3da-4175-b7c0-8da3d8be3ccc None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:52.747430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:52.826216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbef2d-e3da-4175-b7c0-8da3d8be3ccc None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:52.858632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:52.859257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:52.859535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:52.859802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] PATH_INFO: `/v3/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:52.861166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] RBAC: Authorizing `identity:get_role(role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:52.862515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:52.863848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 559/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:52 2022] GET /identity/v3/roles/c17cdb7e94834eee81063576196e3fd9 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:52.870289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:52.968344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cf4fe-087d-4581-82c6-853548a497b1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:53.001126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:53.001697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:53.001980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:53.002255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] PATH_INFO: `/v3/users/0263124817e94b149f7b8ddf3399307a` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:53.003393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] RBAC: Authorizing `identity:get_user(user_id=0263124817e94b149f7b8ddf3399307a)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:53.004833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:53.006645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 562/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:52 2022] GET /identity/v3/users/0263124817e94b149f7b8ddf3399307a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:53.012365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:53.101697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:53.112996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a03f7-515a-4896-af93-ef2c43f1041c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:53.148238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:53.148850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:53.149385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:53.149656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:53.150780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:53.152198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1becd-0d5b-49a6-884c-d69789652a11 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:53.153815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 560/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:53 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:53.159836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:53.194817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:53.262703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a88bcf-5441-4d21-b222-58617e0e21f1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:53.296652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:53.297306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:53.297588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:53.297863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6/users/0263124817e94b149f7b8ddf3399307a/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:53.300032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] RBAC: Authorizing `identity:create_grant(project_id=5d003e81fbd24fa9b283cd4c8535a1b6, user_id=0263124817e94b149f7b8ddf3399307a, role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:53.301468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce04b-0991-4e94-9d41-f106cabbea2e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:53.364744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 563/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:56:53 2022] PUT /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6/users/0263124817e94b149f7b8ddf3399307a/roles/c17cdb7e94834eee81063576196e3fd9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:56:55.247954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260c76b-1ab3-4851-a558-0255ca8ee5ec None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:55.248478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260c76b-1ab3-4851-a558-0255ca8ee5ec None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:55.248478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260c76b-1ab3-4851-a558-0255ca8ee5ec None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:55.249824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 561/1124] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:55.254940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:55.255341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:55.255629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:55.275067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:55.276266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:55.342458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 564/1125] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:55.608527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260c76b-1ab3-4851-a558-0255ca8ee5ec None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:55.707268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260c76b-1ab3-4851-a558-0255ca8ee5ec None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:55.739839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:55.740482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:55.740760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:55.741031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] PATH_INFO: `/v3/roles/c17cdb7e94834eee81063576196e3fd9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:55.742413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] RBAC: Authorizing `identity:get_role(role_id=c17cdb7e94834eee81063576196e3fd9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:55.743813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:55.745103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 562/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:55 2022] GET /identity/v3/roles/c17cdb7e94834eee81063576196e3fd9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:55.750963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:55.772282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:55.849198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25208e9d-850a-473a-8819-16ffc6c69d69 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:55.882964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:55.883527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:55.883809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:55.884083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] PATH_INFO: `/v3/users/0263124817e94b149f7b8ddf3399307a` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:55.885181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] RBAC: Authorizing `identity:get_user(user_id=0263124817e94b149f7b8ddf3399307a)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:55.886604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:55.888184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 565/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:56:55 2022] GET /identity/v3/users/0263124817e94b149f7b8ddf3399307a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:55.893977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:55.920545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:55.993781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3a7e-1ae2-4ab9-ace1-8fd3603d5d29 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:56.027153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:56.027743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:56.028019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:56.028288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:56.094429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:56.095917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:56.119168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:56.191902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 563/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:56:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:56:56.198411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:56.283645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:56.295833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5bb20-d3cf-4b8d-9ad1-b7b90ede4bdc None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:56.329216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:56.329841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:56.330126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:56.330405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] PATH_INFO: `/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:56.331559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] RBAC: Authorizing `identity:get_project(project_id=5d003e81fbd24fa9b283cd4c8535a1b6)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:56.332990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6a28a-1a94-414b-9d11-5ca994480a31 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:56.334880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 566/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:56:56 2022] GET /identity/v3/projects/5d003e81fbd24fa9b283cd4c8535a1b6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:56.341019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:56.437975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe9e0-43f9-411f-a737-2fd16b4d2709 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:56.461080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:56.471173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:56.471767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:56.472049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:56.472320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:56.473044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] RBAC: Adding query filter params (role.id=c17cdb7e94834eee81063576196e3fd9, scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6, user.id=0263124817e94b149f7b8ddf3399307a) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:56:56.473350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:56.475526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7804b3c-b529-4989-94ff-d0c95a4416a6 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:56.489657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 564/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:56:56 2022] GET /identity/v3/role_assignments?user.id=0263124817e94b149f7b8ddf3399307a&scope.project.id=5d003e81fbd24fa9b283cd4c8535a1b6&role.id=c17cdb7e94834eee81063576196e3fd9 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:58.398289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e222-1488-4a96-9fd7-58b65cba4c98 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:58.398889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e222-1488-4a96-9fd7-58b65cba4c98 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:58.398889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e222-1488-4a96-9fd7-58b65cba4c98 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:58.400470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 567/1131] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:56:58.405653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:58.406030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:58.406196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:58.423775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:56:58.424939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:56:58.488449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 565/1132] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:56:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:56:58.757609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e222-1488-4a96-9fd7-58b65cba4c98 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:58.856744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e222-1488-4a96-9fd7-58b65cba4c98 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:58.890195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:58.890856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:58.891149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:58.891431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:58.892561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:58.894012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214be8-48db-477a-89c1-2473d1051447 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:58.895791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 568/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:56:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:56:58.902132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:56:58.997918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816071e1-f9a5-4a0f-9770-efc5afe1a39e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:56:59.015472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:56:59.032106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:56:59.032704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:56:59.032985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:56:59.033259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:56:59.033896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:56:59.035351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4c07-8260-40a7-9318-4af3391f43a5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:56:59.149768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 566/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:56:58 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:57:01.072915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e21a-426a-4d1e-abaa-9958886defd8 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:01.073467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e21a-426a-4d1e-abaa-9958886defd8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:01.073467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415e21a-426a-4d1e-abaa-9958886defd8 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:01.074826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 569/1135] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:01.080210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be203785-275c-433e-af5a-4216b5dd157a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:01.080601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be203785-275c-433e-af5a-4216b5dd157a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:01.080760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be203785-275c-433e-af5a-4216b5dd157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:01.099663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-be203785-275c-433e-af5a-4216b5dd157a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:01.100837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-be203785-275c-433e-af5a-4216b5dd157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:01.167767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 567/1136] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:01.446336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415e21a-426a-4d1e-abaa-9958886defd8 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:01.471059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:01.544970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415e21a-426a-4d1e-abaa-9958886defd8 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:01.577551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:01.578163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:01.578445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:01.578745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:01.579837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:01.581260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d72d0-8732-4e59-8315-5dce772b91ca None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:01.582861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 570/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:57:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:01.588696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be203785-275c-433e-af5a-4216b5dd157a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:01.683973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:01.686887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be203785-275c-433e-af5a-4216b5dd157a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:01.720109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:01.720652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:01.721102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:01.721807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:01.722434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:01.724057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35a374-0b03-443b-9565-6b534ddd37ea None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:01.834969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 568/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:57:01 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:57:03.786449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4cd46-7f3f-4b34-9457-480a1d61ad23 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:03.787048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4cd46-7f3f-4b34-9457-480a1d61ad23 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:03.787048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4cd46-7f3f-4b34-9457-480a1d61ad23 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:03.788380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 571/1139] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:03.793483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:03.793914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:03.794018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:03.812383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:03.813558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:03.878075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 569/1140] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:04.145884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4cd46-7f3f-4b34-9457-480a1d61ad23 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:04.237063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:04.247089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4cd46-7f3f-4b34-9457-480a1d61ad23 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:04.281234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:04.281856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:04.282141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:04.282418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:04.292817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:04.297491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:04.298695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:04.299543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 572/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:04.305283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:04.388621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:04.403859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11959788-11f9-45b8-a0a9-9754a3f11d5d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:04.436197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:04.441112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:04.448050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 570/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:04.453951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:04.554126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b954c-fe02-4b95-8744-f66bd94d93f5 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:04.586524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:04.587181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:04.587488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:04.587767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] PATH_INFO: `/v3/users/cfb8fd100465450cab708738905e915b` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:04.601115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb8fd100465450cab708738905e915b)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:04.602566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:04.604184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 573/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:04 2022] GET /identity/v3/users/cfb8fd100465450cab708738905e915b => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:04.610036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:04.707867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9c5e6-d063-4d21-8454-1a57ff635db2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:04.742135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:04.742762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:04.743049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:04.743325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:04.760702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:04.809098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:04.810570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:04.911936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 571/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:57:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:57:04.917879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:04.930451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:05.016619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e903004-115b-4c9e-a9b0-39916428f9e4 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:05.049667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:05.050286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:05.050569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:05.050873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] PATH_INFO: `/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:05.052887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] RBAC: Authorizing `identity:get_project(project_id=1ca8f6e802cb479699d64ed70d92ebb2)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:05.054338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e7ef81-3d1c-40f3-bff8-e9089d3633e3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:05.056006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 574/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:04 2022] GET /identity/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:05.062305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:05.110062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:05.162123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8c7e6-61d4-4b80-b375-8397dea06e72 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:05.194677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:05.195261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:05.195539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:05.195809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:05.196521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=1ca8f6e802cb479699d64ed70d92ebb2, user.id=cfb8fd100465450cab708738905e915b) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:05.196824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:05.198240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cdea9-defa-453e-a2ba-e2426804e072 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:05.203756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 572/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:57:05 2022] GET /identity/v3/role_assignments?user.id=cfb8fd100465450cab708738905e915b&scope.project.id=1ca8f6e802cb479699d64ed70d92ebb2&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:07.081204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ce36-cec6-4a99-9141-7ee709681237 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:07.081756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ce36-cec6-4a99-9141-7ee709681237 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:07.081756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ce36-cec6-4a99-9141-7ee709681237 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:07.083709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 575/1147] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:07.088714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:07.089137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:07.089270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:07.107514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:07.108707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:07.174570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 573/1148] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:07.445339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021ce36-cec6-4a99-9141-7ee709681237 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:07.523003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:07.544847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021ce36-cec6-4a99-9141-7ee709681237 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:07.577458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:07.578074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:07.578362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:07.578655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:07.588822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:07.593934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:07.595050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:07.596272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 576/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:07.601600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:07.645916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:07.701315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddce39-7586-4f33-8427-f79f3464574d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:07.734096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:07.734715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:07.734994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:07.735265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:07.735923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:07.736222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:07.737581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:07.745872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 574/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:07.751733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:07.849391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75786e90-6e00-4564-8ab2-8047b3cf5299 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:07.882924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:07.883550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:07.883837 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:07.884117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] PATH_INFO: `/v3/users/cfb8fd100465450cab708738905e915b` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:07.885238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] RBAC: Authorizing `identity:get_user(user_id=cfb8fd100465450cab708738905e915b)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:07.886709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:07.888272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 577/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:07 2022] GET /identity/v3/users/cfb8fd100465450cab708738905e915b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:07.893987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:07.992379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccc6ea-f9c6-43f6-8210-9f95d568d32d None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:07.994693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:08.026499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:08.027150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:08.027432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:08.027707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] PATH_INFO: `/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:08.028812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] RBAC: Authorizing `identity:get_project(project_id=1ca8f6e802cb479699d64ed70d92ebb2)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:08.030236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9d161-1581-4934-bfec-94b857fdedbc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:08.032171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 575/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:07 2022] GET /identity/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:08.038567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:08.048403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:08.137024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a236d-4234-40ca-ad8d-405e563a660d None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:08.170380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:08.171059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:08.171345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:08.171623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] PATH_INFO: `/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2/users/cfb8fd100465450cab708738905e915b/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:08.173492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] RBAC: Authorizing `identity:create_grant(project_id=1ca8f6e802cb479699d64ed70d92ebb2, user_id=cfb8fd100465450cab708738905e915b, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:08.174943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d990e5-3c05-4e9a-a25f-b04c33e5767e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:08.261845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 578/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:57:08 2022] PUT /identity/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2/users/cfb8fd100465450cab708738905e915b/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:57:10.246209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366311-bc67-4cca-9624-27c5db103eef None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:10.246786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366311-bc67-4cca-9624-27c5db103eef None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:10.246786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1366311-bc67-4cca-9624-27c5db103eef None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:10.248122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 576/1154] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:10.253213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:10.253632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:10.253777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:10.272611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:10.273795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:10.336150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:10.343430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 579/1155] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:10.614581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366311-bc67-4cca-9624-27c5db103eef None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:10.715832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1366311-bc67-4cca-9624-27c5db103eef None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:10.748373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:10.748979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:10.749256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:10.749525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:10.761300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:10.765984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:10.767100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:57:10.768089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 577/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:10.773283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:10.871340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d260d3b-768c-4788-9a57-1ee463caf612 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:10.905225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:10.905831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:10.906114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:10.906390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:10.907089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:10.907393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:10.908771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:10.916955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 580/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:10.922692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:10.961005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:11.029966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b33b4-c1b8-4eac-b710-88a806666c50 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:11.063271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:11.063887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:11.064164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:11.064431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] PATH_INFO: `/v3/users/cfb8fd100465450cab708738905e915b` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:11.065528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb8fd100465450cab708738905e915b)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:11.067110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:11.068864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 578/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:10 2022] GET /identity/v3/users/cfb8fd100465450cab708738905e915b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:11.074971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:11.133487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:11.174420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd95073-7f6f-4a91-b667-58f5e57576f6 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:11.206948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:11.208169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:11.208528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:11.208732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:11.274229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:11.275718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:11.330363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:11.371806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 581/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:57:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:57:11.377700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:11.471255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:11.474336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0e016-1531-450c-b1e7-d13537aeae72 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:11.506289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:11.506927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:11.507205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:11.507472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] PATH_INFO: `/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:11.508566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] RBAC: Authorizing `identity:get_project(project_id=1ca8f6e802cb479699d64ed70d92ebb2)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:11.509981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826deb-f54a-47dd-a4bd-c98523baaaa7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:11.511594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 579/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:11 2022] GET /identity/v3/projects/1ca8f6e802cb479699d64ed70d92ebb2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:11.517707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:11.614738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6065a2-c41c-46e7-99d1-a0df5ef31d56 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:11.647536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:11.648133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:11.648416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:11.648691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:11.649671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=1ca8f6e802cb479699d64ed70d92ebb2, user.id=cfb8fd100465450cab708738905e915b) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:11.649981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:11.651442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd342a-0420-49a0-998a-818d1d3726dd None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:11.657064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 582/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:57:11 2022] GET /identity/v3/role_assignments?user.id=cfb8fd100465450cab708738905e915b&scope.project.id=1ca8f6e802cb479699d64ed70d92ebb2&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:13.573019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5632e-2e7d-4d54-b958-b992a8c475b2 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:13.573583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5632e-2e7d-4d54-b958-b992a8c475b2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:13.573583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5632e-2e7d-4d54-b958-b992a8c475b2 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:13.574990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 580/1162] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:13.580084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:13.580477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:13.580644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:13.598985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:13.600172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:13.616246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:13.665802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 583/1163] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:13.933618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5632e-2e7d-4d54-b958-b992a8c475b2 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:14.034030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5632e-2e7d-4d54-b958-b992a8c475b2 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:14.069087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:14.069684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:14.069964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:14.070238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:14.103345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:14.135829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:14.137348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:14.234309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 581/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:57:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:57:14.240850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:14.337400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2480d5-4b89-4c6e-a802-2379ac797710 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:14.370872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481402b-dbff-4399-b504-ee797769852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:14.371498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481402b-dbff-4399-b504-ee797769852b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:14.371780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481402b-dbff-4399-b504-ee797769852b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:14.372055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481402b-dbff-4399-b504-ee797769852b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:14.378794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481402b-dbff-4399-b504-ee797769852b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:14.380405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481402b-dbff-4399-b504-ee797769852b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-a481402b-dbff-4399-b504-ee797769852b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:14.391305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:14.392558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:14.393460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 584/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:57:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:14.399371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:14.463471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:14.495452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58379c81-90d9-4c35-8dbc-2592da47b317 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:14.529155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:14.529750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:14.530034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:14.530311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:14.530981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:14.531288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:14.532648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d0199-01b9-4d42-8840-debea3f5e345 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:14.540158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 582/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Aug 26 05:57:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:16.435152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c1fa-e444-4ea5-a4e6-0a84cf887a37 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:16.435707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c1fa-e444-4ea5-a4e6-0a84cf887a37 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:16.435707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958c1fa-e444-4ea5-a4e6-0a84cf887a37 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:16.437033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 585/1167] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:16.443218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1cd1-22ad-4ab4-96a7-300582f9f7db None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:16.443644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1cd1-22ad-4ab4-96a7-300582f9f7db None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:16.443768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb1cd1-22ad-4ab4-96a7-300582f9f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:16.465900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-58bb1cd1-22ad-4ab4-96a7-300582f9f7db None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:16.467096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-58bb1cd1-22ad-4ab4-96a7-300582f9f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:16.532043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 583/1168] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:16.802041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958c1fa-e444-4ea5-a4e6-0a84cf887a37 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:16.817065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:16.903545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958c1fa-e444-4ea5-a4e6-0a84cf887a37 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:16.937841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:16.938438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:16.938748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:16.939026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:16.939676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:16.941027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15858f3d-6f07-47de-98d7-d0686b38d15b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:17.016724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 586/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Fri Aug 26 05:57:16 2022] POST /identity/v3/domains => generated 267 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:57:18.935153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252b7-7f86-4577-9258-125eb8761d7a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:18.935646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252b7-7f86-4577-9258-125eb8761d7a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:18.935748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784252b7-7f86-4577-9258-125eb8761d7a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:18.937019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 584/1170] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:18.943289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:18.943705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:18.943848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:18.961825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:18.963034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:19.029058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 587/1171] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:19.300039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784252b7-7f86-4577-9258-125eb8761d7a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:19.360379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:19.397441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784252b7-7f86-4577-9258-125eb8761d7a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:19.431234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:19.431846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:19.432129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:19.432394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] PATH_INFO: `/v3/domains/6174030a343744118ac4c8bd68baf933` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:19.442885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=6174030a343744118ac4c8bd68baf933)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:19.444322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2a544-8b3e-4cfe-94d5-eeac2710f4fe None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:19.445880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 585/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 26 05:57:19 2022] GET /identity/v3/domains/6174030a343744118ac4c8bd68baf933 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:19.451672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:19.453879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:19.549106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bdb2d-f636-400b-8215-33a87a0df9a5 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:19.582425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:19.583004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:19.583290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:19.583569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:19.584220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:19.585661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a6618-6e7d-427d-a938-7d02b0fc2d00 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:19.674900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 588/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:57:19 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:57:21.614145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f344-1e8a-4183-9cf5-8855da9a2ffb None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:21.614768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f344-1e8a-4183-9cf5-8855da9a2ffb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:21.614768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f344-1e8a-4183-9cf5-8855da9a2ffb None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:21.616088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 586/1174] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:21.621392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:21.621826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:21.621960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:21.640284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:21.641467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:21.690375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:21.708749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 589/1175] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:57:21.981341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3f344-1e8a-4183-9cf5-8855da9a2ffb None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:22.082268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3f344-1e8a-4183-9cf5-8855da9a2ffb None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:22.103521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:22.115028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] PATH_INFO: `/v3/domains/6174030a343744118ac4c8bd68baf933` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6174030a343744118ac4c8bd68baf933)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2320ae9-72ff-4457-ae1b-b2c00f3d2522 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:22.124771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 587/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Aug 26 05:57:21 2022] GET /identity/v3/domains/6174030a343744118ac4c8bd68baf933 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:22.126177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:22.230313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c535a-84e1-4183-83b6-a71747dcf70a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:22.262862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:22.263473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:22.263757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:22.264032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:22.264681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:22.266110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6d67c-ad59-4d80-b09d-e6c5601d854c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:22.377076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 590/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:57:22 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 05:57:24.324623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94f4a-071f-4a53-94f7-03b3205ed02d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:24.325172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94f4a-071f-4a53-94f7-03b3205ed02d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:24.325172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f94f4a-071f-4a53-94f7-03b3205ed02d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:24.326484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 588/1178] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:24.331707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:24.332104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:24.332260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:24.350312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:24.351524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:24.385444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:24.416216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 591/1179] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:57:24.684295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f94f4a-071f-4a53-94f7-03b3205ed02d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:24.782845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f94f4a-071f-4a53-94f7-03b3205ed02d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:24.814108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:24.819774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:24.819774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:24.819774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:24.826223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:24.830830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:24.831968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:57:24.833788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 589/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:24.838906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:24.940112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0271fc8-4ba5-4c74-9662-24b2544fde16 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:24.973304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:24.974145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:24.974427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:24.974736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:24.975403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:24.982341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:24.982341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:24.984823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 592/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:24.991304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:25.040366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:25.090165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d83181-5cbf-4758-8b0a-095870bc35c9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:25.122689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:25.123308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:25.123594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:25.123874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] PATH_INFO: `/v3/users/b60999011f0e4fab9b8d332b03ff8985` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:25.137194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b60999011f0e4fab9b8d332b03ff8985)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:25.138645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:25.140216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 590/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:24 2022] GET /identity/v3/users/b60999011f0e4fab9b8d332b03ff8985 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:25.146378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:25.177868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:25.244669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea8631-87d1-41b7-a3ec-597697352b95 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:25.277798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:25.278393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:25.278702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:25.278976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:25.344411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:25.345899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:25.375445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:25.443857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 593/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:57:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:57:25.449893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:25.546367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536749a1-493a-4cb4-9a53-3e902d24a0f3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:25.555783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:25.579588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:25.580206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:25.580487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:25.580763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] PATH_INFO: `/v3/projects/d03cde40014b4d61bbb659472ccf202c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:25.582143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=d03cde40014b4d61bbb659472ccf202c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:25.584195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f3726-5adb-478d-b730-d57f6f332ea4 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:25.585903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 591/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:25 2022] GET /identity/v3/projects/d03cde40014b4d61bbb659472ccf202c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:25.591925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:25.688966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684229b-6981-4419-9379-ddfa5829d420 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:25.717390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:25.722187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:25.725825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:25.725825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:25.725825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:25.725825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=d03cde40014b4d61bbb659472ccf202c, user.id=b60999011f0e4fab9b8d332b03ff8985) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:25.725825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:25.726332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6115f7c-2556-49a3-b11b-f62c9688478d None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:25.731254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 594/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:57:25 2022] GET /identity/v3/role_assignments?user.id=b60999011f0e4fab9b8d332b03ff8985&scope.project.id=d03cde40014b4d61bbb659472ccf202c&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:27.607460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb463d-4a4a-4807-b8ad-d63b4d7940ec None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:27.607992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb463d-4a4a-4807-b8ad-d63b4d7940ec None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:27.608086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb463d-4a4a-4807-b8ad-d63b4d7940ec None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:27.609374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 592/1186] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:27.614514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:27.614968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:27.615085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:27.633284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:27.634471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:27.699332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 595/1187] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:27.970844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb463d-4a4a-4807-b8ad-d63b4d7940ec None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:28.068628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb463d-4a4a-4807-b8ad-d63b4d7940ec None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:28.100177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:28.100780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:28.101056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:28.101326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:28.110915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:28.116054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:28.117162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:57:28.118377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 593/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:28.123062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:28.218926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66799f04-7e04-4347-ae7a-f02bbd32c7a1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:28.244239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:28.252659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:28.253262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:28.257791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:28.257791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:28.257791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:28.257791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:28.257791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:28.264553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 596/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:28.270945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:28.306574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:28.370919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a7f81-0b37-47ab-9763-99676ea8fb1c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:28.404829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:28.405441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:28.405723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:28.406002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] PATH_INFO: `/v3/users/b60999011f0e4fab9b8d332b03ff8985` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:28.407133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] RBAC: Authorizing `identity:get_user(user_id=b60999011f0e4fab9b8d332b03ff8985)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:28.408564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:28.410118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 594/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:28 2022] GET /identity/v3/users/b60999011f0e4fab9b8d332b03ff8985 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:28.416095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:28.514022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914d371-f7ca-403e-8e8f-b27501010475 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:28.546889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:28.547508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:28.547794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:28.548071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] PATH_INFO: `/v3/projects/d03cde40014b4d61bbb659472ccf202c` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:28.549183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] RBAC: Authorizing `identity:get_project(project_id=d03cde40014b4d61bbb659472ccf202c)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:28.550647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6758c-8755-4d90-8871-0d62b725abbf None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:28.552480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 597/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:28 2022] GET /identity/v3/projects/d03cde40014b4d61bbb659472ccf202c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:28.558301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:28.656135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460b513-5e97-48d4-a821-e2c9acb9aa89 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:28.658397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:28.688852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:28.689574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:28.689860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:28.690136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] PATH_INFO: `/v3/projects/d03cde40014b4d61bbb659472ccf202c/users/b60999011f0e4fab9b8d332b03ff8985/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:28.692140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=d03cde40014b4d61bbb659472ccf202c, user_id=b60999011f0e4fab9b8d332b03ff8985, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:28.694239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62536c-9fe1-416d-a90f-64c4a86d1ffb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:28.770441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 595/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:57:28 2022] PUT /identity/v3/projects/d03cde40014b4d61bbb659472ccf202c/users/b60999011f0e4fab9b8d332b03ff8985/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:57:30.672109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e5b9-0c75-49a0-ab04-46f4d0a6568c None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:30.672638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e5b9-0c75-49a0-ab04-46f4d0a6568c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:30.672638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e5b9-0c75-49a0-ab04-46f4d0a6568c None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:30.673973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 598/1193] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:57:30.679776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:30.680197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:30.680316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:30.697830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:57:30.699049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:57:30.763730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 596/1194] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:57:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:57:31.033069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e5b9-0c75-49a0-ab04-46f4d0a6568c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.090262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:31.134340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e5b9-0c75-49a0-ab04-46f4d0a6568c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:31.167799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:31.168419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:31.168701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:31.168974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:31.179989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:57:31.184612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:57:31.185726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:57:31.186967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 599/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:57:31.191782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.282143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:31.289400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17cca7-9341-4d5c-b339-cbb961d92db3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:31.322595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:31.323300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:31.323581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:31.323856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:31.324521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:31.324825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:31.326190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:31.333961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 597/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:31.340023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.440969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94319d07-c400-4736-8c03-606d28bdd5e7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:31.456625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:31.474700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:31.475520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:31.475806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:31.476083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] PATH_INFO: `/v3/users/b60999011f0e4fab9b8d332b03ff8985` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:31.477205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] RBAC: Authorizing `identity:get_user(user_id=b60999011f0e4fab9b8d332b03ff8985)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:31.478670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:31.480231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 600/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/users/b60999011f0e4fab9b8d332b03ff8985 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:31.486304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.585341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fcc24-9b4a-47bc-ae04-5b0859e2393b None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:31.618640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:31.619254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:31.619531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:31.619798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:31.643064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:31.685718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:31.687189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:31.783481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 598/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:57:31.789620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.885741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c94e23-a8ea-47c0-8887-abe53ac7b748 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:31.919009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:31.919630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:31.919908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:31.920180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] PATH_INFO: `/v3/projects/d03cde40014b4d61bbb659472ccf202c` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:31.921268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] RBAC: Authorizing `identity:get_project(project_id=d03cde40014b4d61bbb659472ccf202c)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:31.922717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c4fe6-e8f6-4941-8163-8d86c7c682f5 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:31.924328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 601/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/projects/d03cde40014b4d61bbb659472ccf202c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:31.930825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:57:31.985567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:32.028446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9db328-6b50-47c3-be3a-2f6b019e3ba2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:57:32.060697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:57:32.061291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:57:32.061573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:57:32.061866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:57:32.062584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=d03cde40014b4d61bbb659472ccf202c, user.id=b60999011f0e4fab9b8d332b03ff8985) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:57:32.062924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:57:32.064349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fc0c69-e201-4ad0-b681-b14988289edd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:57:32.070666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 599/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:57:31 2022] GET /identity/v3/role_assignments?user.id=b60999011f0e4fab9b8d332b03ff8985&scope.project.id=d03cde40014b4d61bbb659472ccf202c&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:57:41.914828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=74 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:57:42.067979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=40 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:47.121700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8a68-1952-4d5f-816b-60e29de439c0 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:47.122271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8a68-1952-4d5f-816b-60e29de439c0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:47.122271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8a68-1952-4d5f-816b-60e29de439c0 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:47.123636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 602/1201] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:58:47.128802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:47.129200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:47.129351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:47.148946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:58:47.150057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:58:47.215564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 600/1202] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:58:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:58:47.489095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8a68-1952-4d5f-816b-60e29de439c0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:47.595438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8a68-1952-4d5f-816b-60e29de439c0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:47.631394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:47.632642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:47.632923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:47.633196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:47.643320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:47.644845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:47.660006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:47.661143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:47.662143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:47.662143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:47.662143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:47.662143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 603/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:47.667858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:47.766354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e231b1-3995-4ecb-8c4f-c0d98d8b9fbc None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:47.794632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:47.800598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:47.806548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba1fda-1da9-4793-b42e-d5cb3d799e20 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:47.815533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 601/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:47.821592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:47.876736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:47.924103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eca343-6fc0-436e-b3be-7897805accd2 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:47.957451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:47.958056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:47.958339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:47.958681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:47.959330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:47.962708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdfb02-c4b2-4e57-83cc-b251776b3434 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:48.076252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 604/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Aug 26 05:58:47 2022] POST /identity/v3/users => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:58:50.031564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff0d4b-a620-4241-8a55-e4ca124cbd16 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:50.032103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff0d4b-a620-4241-8a55-e4ca124cbd16 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:50.032103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff0d4b-a620-4241-8a55-e4ca124cbd16 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:50.033490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 602/1206] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:58:50.039397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:50.042257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:50.042257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:50.058701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:58:50.059912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:58:50.129962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 605/1207] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:58:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:58:50.405386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff0d4b-a620-4241-8a55-e4ca124cbd16 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:50.508118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff0d4b-a620-4241-8a55-e4ca124cbd16 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:50.542326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:50.543098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:50.543379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:50.543647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:50.554307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:58:50.559646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:50.560818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:50.561887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 603/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:50.566891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:50.610118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:50.670513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc44561-c4c1-4685-bb89-872ca9acdd5e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:50.705590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:50.706193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:50.706472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:50.706781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:50.707443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:50.707742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:50.709103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:50.717296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 606/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:50.723362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:50.773162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:50.821493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba29c-7f51-4c4f-9490-55f126d93cb0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:50.854850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:50.855474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:50.855751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:50.856020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:50.867366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:50.868909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:50.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:50.884681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:50.885778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:50.885778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:50.885778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:50.885778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 604/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:58:50.890322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:50.988489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:50.992124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db536859-f132-434f-97aa-f6f60c06dbfc None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.026112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.026753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.027035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.027311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.027956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:51.028363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.029718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:51.040872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 607/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:51.046769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:51.159715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de9498-26ad-4f79-996e-a3044a93804e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.161280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:51.193508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.194880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.195170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.195645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.203345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.204787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:58:51.216460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:51.217568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:51.218564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 605/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:51.223822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:51.324078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822674-a2fe-4cce-ab6c-a14d3fa6fb9e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.357919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.358558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.358881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.359160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.359887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:51.360197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.361640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:51.378772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 608/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:58:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:51.385414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:51.486211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203d5e9-3540-4bd2-9b38-af38956a3d24 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.519333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.519971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.520247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.520516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.550580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:51.588730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.590103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:51.686151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 606/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:58:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:58:51.692050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:51.711826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:51.789892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb9ccd-84e9-407c-9320-7c63cafc2db0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.824423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.825048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.825330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.825607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.835531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.836969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:51.851249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:51.852385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:51.853285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:51.853285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:51.853285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:51.853285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 609/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:51.858166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:51.919228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:51.957500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1426e-da42-40c6-b73a-d788fb7b5818 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:51.990275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:51.991633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:51.991967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:51.992245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:51.992891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:51.993192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:51.994675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:52.006025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 607/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:52.012067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:52.081285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:52.110368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37b561-8c67-4b86-9f91-8c9a68d8f709 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:52.142794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:52.147855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:52.147855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:52.147855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:52.150516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:52.151974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:58:52.163952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:52.165032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:52.165941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 610/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:58:52.171206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:52.268707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5801d4-03ee-4035-9417-83b1f9f21396 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:52.290770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:52.301624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:52.302214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:52.302487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:52.302869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:52.303732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:52.303984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:52.305410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbebd9-39af-414a-beba-625a8a689a9c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:52.316698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 608/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:52.323738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:52.422832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61b8fe-8527-4293-a639-f66594183a58 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:52.451104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:52.457610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:52.458221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:52.458506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:52.458820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:52.460357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=48e76f49c2224e959267952f79669906) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:52.460780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:52.462225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e1f83-4460-4d50-a064-8a927773af30 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:52.467818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 611/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:58:52 2022] GET /identity/v3/role_assignments?user.id=48e76f49c2224e959267952f79669906&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:54.369517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137cb5e-501b-4b4c-bf23-7b054ef5db73 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:54.370094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137cb5e-501b-4b4c-bf23-7b054ef5db73 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:54.370203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137cb5e-501b-4b4c-bf23-7b054ef5db73 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:54.371561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 609/1220] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:58:54.376667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:54.377081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:54.377277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:54.395826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:58:54.398574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:58:54.467371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 612/1221] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:58:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:58:54.742571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137cb5e-501b-4b4c-bf23-7b054ef5db73 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:54.843000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137cb5e-501b-4b4c-bf23-7b054ef5db73 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:54.876270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:54.876888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:54.877167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:54.877470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:54.888202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:58:54.893012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:54.894060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:54.899035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 610/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:58:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:58:54.900585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.001389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98060977-94ac-4a05-855a-1d3f8f6cbe5d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.025759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:55.036284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.036890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.037173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.037474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.038226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] RBAC: Adding query filter params (name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:55.038531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.039936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.047639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 613/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:58:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:55.054668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.103249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:55.158190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9746645-cd01-4ef1-a1a0-5a4585109d42 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.194191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.194882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.195339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.198040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.213756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.215353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:55.233778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:55.241341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:55.242839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:55.242839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:55.242839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:55.242839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 611/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:55.243144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.362741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fbc94-873f-4560-9bef-93a5b070153c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.397086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.398061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.398223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.398499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.399175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:55.399521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.400930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.413260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 614/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:55.419287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.541362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:55.548122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cadfc4-a1fc-432a-a027-6d70ae983acc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.582302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.587390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.587390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.587390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.591593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.593995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:58:55.605604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:55.606702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:55.607869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:55.610759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 612/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:55.621763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.664307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:55.717076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65410a91-a473-4cc7-9a64-4b535cffa443 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.751918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.752585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.752871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.753150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.753884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:55.754262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.755735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.774231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 615/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:55.780368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:55.885033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dd399-82a6-420b-9366-050c34d9dd17 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:55.918239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:55.918934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:55.919214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:55.919568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:55.931080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:55.932673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:55.947243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:55.949726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:55.950643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:55.950643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:55.950643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:55.950643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 613/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:55.954368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:56.059313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf7274-f4ca-49a2-aa4c-9e46c7c654f9 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:56.061646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:56.092554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:56.093160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:56.093445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:56.093720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:56.094362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:56.094693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:56.096052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:56.107395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 616/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:56.113147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:56.123397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:56.214767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d49d66-ded9-4a2d-a5dc-4017460cf87c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:56.249431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:56.250100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:56.250373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:56.250672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:56.257664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:56.259172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:58:56.270438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:56.273241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:58:56.274362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 614/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:58:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:56.278431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:56.378353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c59d2-7eb0-44d7-917a-e1e64cea1d4c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:56.413051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:56.413886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:56.414174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:56.414484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:56.415213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:56.415525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:56.416972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814e300-871b-476e-b075-5df4954427d8 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:56.429400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 617/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:58:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:56.435443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:56.501546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:56.536579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85199a45-7ef4-4268-9d6f-f90a11494490 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:56.569880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:56.570557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:56.570862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:56.571139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/48e76f49c2224e959267952f79669906/roles/236c7f6fe30b49a6acd545921e587875` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:56.586053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=48e76f49c2224e959267952f79669906, role_id=236c7f6fe30b49a6acd545921e587875)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:56.588402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60af0f-52aa-4996-b5e2-88e6348cd0f7 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:56.675209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 615/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:58:56 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/48e76f49c2224e959267952f79669906/roles/236c7f6fe30b49a6acd545921e587875 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:58:58.587867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc048-ac23-4733-aa13-6e1791f11393 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:58.588435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc048-ac23-4733-aa13-6e1791f11393 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:58.588435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc048-ac23-4733-aa13-6e1791f11393 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:58.589753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 618/1233] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:58:58.595395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:58.595807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:58.595941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:58.613808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:58:58.615036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:58:58.683094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 616/1234] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:58:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:58:58.956107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc048-ac23-4733-aa13-6e1791f11393 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:58.982471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:59.059757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc048-ac23-4733-aa13-6e1791f11393 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.093004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.093623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.093902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.094172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.105362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:58:59.110091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:59.111139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:59.111976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 619/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Aug 26 05:58:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:59.118546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:59.165830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:59.219599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493652b1-0a41-4614-baf8-c8dce6aae7c0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.257151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.257805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.258085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.258353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.259048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:59.259343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:59.260702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:59.268666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 617/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:59.275005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:59.363054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:59.378907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf6c98-3e37-4b21-83ef-980f87681a21 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.415214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.415890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.416148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.416450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.426674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:59.431789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:58:59.442304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:59.443347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:58:59.444892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:58:59.444892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:58:59.444892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:59.444892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 620/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:59.449971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:59.551011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1ab0b-e1b2-404b-a47c-a647258b9c3f None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.559713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:59.584756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.585416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.585690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.585957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.586589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:59.586915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:59.588294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:59.600729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 618/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:59.608004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:59.710771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e1fe-6834-4bda-b4f0-f2bcd4aefb56 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.743900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.744523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.744805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.745080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.747733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:58:59.752507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:59.753950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:58:59.765296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:58:59.769412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:58:59.770550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 621/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:58:59.772705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:58:59.872266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286398-9d93-4a68-aa19-8f0db271f85a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:58:59.906301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:58:59.906942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:58:59.907219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:58:59.907486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:58:59.908202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:58:59.908499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:58:59.909932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:58:59.927322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 619/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:58:59.933426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.037997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d50d4-4fb5-4070-9d61-86911b86f2c7 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:00.072291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:00.072896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:00.073179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:00.073453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:00.137505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:00.143849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:00.145311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:00.242271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 622/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:58:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:59:00.248261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.276349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:00.347519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b8966-f7f0-48a9-bfd7-6b5973aaf8e5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:00.381385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:00.382108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:00.382432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:00.382732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:00.392669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:00.394153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:00.410332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:00.411388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:00.412905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:00.412905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:00.412905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:00.412905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 620/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:00.417898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.512407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:00.518206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0a841-3269-4b70-8c8e-9783fa0bba4e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:00.552545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:00.553916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:00.554269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:00.554538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:00.555218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:00.555526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:00.556904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:00.568078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 623/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:00.574283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.653249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:00.673571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32427c2-e462-43e0-8ac0-158aa2412e22 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:00.705793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:00.710900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:00.710900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:00.710900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:00.713302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:00.714805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:59:00.725579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:00.726689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:00.727796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 621/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:59:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:00.732776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.834857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b6a63-4d28-4d92-9db5-d329f9978f86 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:00.869694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:00.870303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:00.870589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:00.870898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:00.872218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:00.872535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:00.874007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0e29d-6acd-4036-ba45-dfb1caf4812b None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:00.887346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 624/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:59:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:00.893708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:00.995258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ecbf8-3f51-49d8-876a-0b5e5d5a3502 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:01.028139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:01.028728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:01.028999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:01.029267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:01.029977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] RBAC: Adding query filter params (role.id=236c7f6fe30b49a6acd545921e587875, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=48e76f49c2224e959267952f79669906) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:01.030278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:01.031732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7393e-de6a-42cd-a9cd-fc7e76da7e79 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:01.037784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 622/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:59:00 2022] GET /identity/v3/role_assignments?user.id=48e76f49c2224e959267952f79669906&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=236c7f6fe30b49a6acd545921e587875 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:03.057787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349dbe-6c55-43d4-8543-f21040ebb660 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:03.058323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349dbe-6c55-43d4-8543-f21040ebb660 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:03.058429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349dbe-6c55-43d4-8543-f21040ebb660 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:03.059753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 625/1247] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:03.065018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:03.065415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:03.065557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:03.072711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:03.085299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:03.090213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:03.154785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 623/1248] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 05:59:03.438107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec349dbe-6c55-43d4-8543-f21040ebb660 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:03.456728 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:03.540171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec349dbe-6c55-43d4-8543-f21040ebb660 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:03.573237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:03.573862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:03.574145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:03.574421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:03.584763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:59:03.589388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:03.590509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:03.591576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 626/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:03.596622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:03.695330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372acbb2-91b1-4888-9771-e9acf18dd2f5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:03.723401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:03.728622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:03.729221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:03.729502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:03.729776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:03.730429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:03.730760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:03.732120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:03.739893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 624/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:03.746182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:03.824479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:03.846865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91352667-36d4-4a47-ad8d-9959ed824d02 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:03.882106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:03.882797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:03.883083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:03.883362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:03.895834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:03.897921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:03.912747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:03.913922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:03.915290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:03.915290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:03.915290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:03.915290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 627/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:03.919875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.019581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d25fbd-78b0-4843-9157-fbfb5d51b4a3 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:04.053284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:04.053915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:04.054197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:04.054497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:04.055206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:04.055517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:04.056937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:04.069162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 625/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:04.075090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.179397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d7e41-1c21-4af8-bcc7-3afd24c30ed1 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:04.205753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:04.213816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:04.214442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:04.214767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:04.215045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:04.222303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:04.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:59:04.234838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:04.235862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:04.236792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 628/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:04.242512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.297337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:04.340016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4d874-1ef6-414d-9b40-2c073910c10c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:04.372752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:04.373359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:04.373639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:04.373913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:04.374658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:04.374969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:04.376404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:04.394318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 626/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:59:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:04.400537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.500139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd284-041b-4588-8f1a-b391be7c0c58 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:04.534132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:04.534770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:04.535055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:04.535333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:04.590747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:04.603396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:04.604862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:04.703732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 629/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:59:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:59:04.710234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.808181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7f82b-9a98-4528-992d-2ff9b19d6e3c None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:04.821453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:04.841762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:04.842319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:04.842594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:04.842896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:04.852469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:04.853899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:04.867550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:04.868661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:04.869559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:04.869559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:04.869559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:04.869559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 627/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:59:04.874473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:04.961335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:04.973151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9162916-0e69-4445-a3bb-3079b272e7a3 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:05.007867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:05.008469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:05.008751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:05.009015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:05.009663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:05.009963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:05.011436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:05.022817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 630/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:05.028511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:05.127326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1ab90-e2bb-4eae-9603-7a81cb7261f9 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:05.161677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:05.162697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:05.162982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:05.163255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:05.170454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:05.172601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:59:05.185039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:05.186146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:05.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 628/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:05.192171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:05.294221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72affe3-30d8-4154-9a98-579b69f35a4c None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:05.327798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:05.328403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:05.328690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:05.328969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:05.329655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:05.329965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:05.331496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef5dc-420c-486f-b0fd-b6447dc6bd75 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:05.343289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 631/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:05.350518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:05.376718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:05.453968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a97d9-8469-4952-b8c2-aec6c0d966bc None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:05.489285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:05.489888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:05.491205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:05.491523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:05.491978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=48e76f49c2224e959267952f79669906) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:05.492284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:05.493711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7df20e-44b6-40be-85a7-02d4c108f076 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:05.500319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 629/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:59:05 2022] GET /identity/v3/role_assignments?user.id=48e76f49c2224e959267952f79669906&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:07.391116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeacc5-a3e8-4585-9f89-a2f9e3e35da1 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:07.391698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeacc5-a3e8-4585-9f89-a2f9e3e35da1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:07.391698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeacc5-a3e8-4585-9f89-a2f9e3e35da1 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:07.393049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 632/1261] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:07.399270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:07.399674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:07.399818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:07.418533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:07.420289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:07.487510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 630/1262] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:07.759383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaeacc5-a3e8-4585-9f89-a2f9e3e35da1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:07.770298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:07.863039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaeacc5-a3e8-4585-9f89-a2f9e3e35da1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:07.896863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:07.897788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:07.898075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:07.898353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:07.908901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:59:07.913897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:07.915020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:07.915877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 633/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:07.921010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:07.939397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:08.018750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd9b6e-76a1-47d3-90c5-341e1692fd19 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.051624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.052223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.053031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.053443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.054110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:08.054417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.055822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.063730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 631/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:08.070324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:08.144673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:08.177043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dae0ea-ac12-4584-9f98-52cd42f5d0f0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.211974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.212596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.212881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.213157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.223021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.224503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:08.241582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:08.242750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:08.243841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:08.243841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:08.243841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:08.243841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 634/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:08.248731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:08.332705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:08.351204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfa19e-03b6-4c6c-9ab0-e8d141640795 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.384154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.384757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.385039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.385315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.385958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:08.386262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.387673 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.399575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 632/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:08.405766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:08.511235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f3ecf-cd7a-431b-ae4a-4e25e47830aa None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.532092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:08.548558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.549187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.549470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.549742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] PATH_INFO: `/v3/users/placement` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.558502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.560094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:59:08.576082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:08.577078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:08.578027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 635/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:08.582317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:08.682332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fa58a-32b4-480f-933d-a0ef0002a1cb None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.716267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.716869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.717147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.717421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.718138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:08.718440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.719936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.729083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:08.737206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 633/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:08.743415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:08.842129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879306d8-ca5e-43c0-8ee6-d0c70238ebea None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:08.877604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:08.878216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:08.878499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:08.878801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:08.889232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:08.890729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:08.906026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:08.916491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:08.917575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:08.917575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:08.917575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:08.917575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 636/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:08.917853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:09.012913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da587a-231d-491c-b154-f23c8f6268d5 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:09.045903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:09.050069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:09.060666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 634/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:09.066769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:09.099672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:09.168386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d38ec0-82ca-49fa-8cf2-5e898f7fd3b0 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:09.202678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:09.203319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:09.203603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:09.203878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:09.211399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:09.212842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:59:09.223790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:09.224838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:09.225904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:09.226952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 637/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:09.232911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:09.290451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:09.329468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6180d-b40e-4192-83c3-3e72b370608e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:09.361941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:09.362544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:09.362855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:09.363132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:09.363871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:09.364174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:09.365660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b960bb-feec-4aa9-8e21-5667b300c7d2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:09.377265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 635/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:09.384071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:09.477622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:09.486371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdb253-7a01-449a-a533-63f8f684c547 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:09.521490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:09.521883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:09.522170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:09.522748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] PATH_INFO: `/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/48e76f49c2224e959267952f79669906/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:09.524632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] RBAC: Authorizing `identity:create_grant(project_id=cdab9db7f04e4686b082a3ca17fb9d07, user_id=48e76f49c2224e959267952f79669906, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:09.526069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623835b-4608-454e-9385-9a6d0f501bae None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:09.624171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 638/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Aug 26 05:59:09 2022] PUT /identity/v3/projects/cdab9db7f04e4686b082a3ca17fb9d07/users/48e76f49c2224e959267952f79669906/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 05:59:11.590910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e05dc-4292-4243-8d1a-4331fd24be71 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:11.591471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e05dc-4292-4243-8d1a-4331fd24be71 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:11.591471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e05dc-4292-4243-8d1a-4331fd24be71 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:11.592787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 636/1274] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:11.598361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:11.598824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:11.598961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:11.617459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:11.618676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:11.690031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 639/1275] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:11.961212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e05dc-4292-4243-8d1a-4331fd24be71 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.069365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e05dc-4292-4243-8d1a-4331fd24be71 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.088620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:12.109377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.110017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.110304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.110577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.121576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 05:59:12.126229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:12.127350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:12.128395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 637/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Aug 26 05:59:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 05:59:12.133706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.214338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:12.232739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e4b4c-5a77-4eeb-b3a6-46d4ead3b93f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.267681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.268299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.268586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.268868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.269660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:12.269848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:12.271290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:12.279715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 640/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:12.285842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.385709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7787b-82b4-48f5-a830-f0776a145a99 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.418730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.419342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.419616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.419884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.430095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:12.431564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:12.445057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:12.446329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:12.447451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:12.447451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:12.447451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:12.447451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 638/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:12.452091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.552129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a178d-b4dd-4556-afc6-87462bdaa3ba None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.585957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.586557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.586869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.587146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.587787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:12.588091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:12.589453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:12.600850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 641/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:12.606580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.638391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:12.705273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67606679-d52e-4a7b-a41e-8711997b8f7a None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.738289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.738933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.739267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.739544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.747432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:12.748941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 05:59:12.760238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:12.761442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:12.762649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 639/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:12.767177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:12.771107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:12.868016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0729-bfa3-4048-abc9-d9dbae59cc8f None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:12.902878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:12.903485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:12.903770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:12.904049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:12.904781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:12.905093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:12.906673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:12.924659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 642/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:12.930333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.018271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:13.028414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba2862-335a-4dea-9bf3-08efd56ed5e2 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:13.060042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:13.060634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:13.060913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:13.061178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:13.129403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:13.131325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:13.218282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:13.228400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 640/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Aug 26 05:59:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 05:59:13.234192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.295627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:13.333062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712582-d7b8-4b30-a753-c63c0b3ab96e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:13.365634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:13.366244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:13.366522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:13.366823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:13.376271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:13.377704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 05:59:13.392190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:13.393345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 05:59:13.394437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 05:59:13.394437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 05:59:13.394437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:13.394437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 643/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 05:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:13.399263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.495962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abc84-b3ee-4986-9670-9f3d14398ee8 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:13.530413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:13.531052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:13.531328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:13.531593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:13.532219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:13.532515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:13.533866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:13.545526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 641/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Aug 26 05:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:13.551432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.651863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170861-f8ce-452d-805a-9410a13f851e None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:13.665622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:13.685100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:13.685731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:13.686017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:13.686295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:13.693414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:13.694877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 05:59:13.705886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:13.713403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: ERROR keystone.server.flask.application Aug 26 05:59:13.714455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 644/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Aug 26 05:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:13.715116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.763651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:13.811714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa16d8d-8364-4626-998e-721cb52e04d1 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:13.846741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:13.847390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:13.847721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:13.847997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:13.848685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:13.848993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:13.850648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225d1d0-fcee-4050-aeb3-a29237e53553 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:13.862348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 642/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Aug 26 05:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:13.868736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:13.967401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a07fda-c601-403b-a4b1-34a878e64b13 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:14.002098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:14.002724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:14.003007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:14.003278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:14.003996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] RBAC: Adding query filter params (role.id=b4c6ab88b8d64fcdacfc6c5268120e8d, scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07, user.id=48e76f49c2224e959267952f79669906) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:14.004303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:14.006402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96a1b-13c0-4d92-a0d0-899139117f53 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:14.012120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 645/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Aug 26 05:59:13 2022] GET /identity/v3/role_assignments?user.id=48e76f49c2224e959267952f79669906&scope.project.id=cdab9db7f04e4686b082a3ca17fb9d07&role.id=b4c6ab88b8d64fcdacfc6c5268120e8d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:15.950916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45cf-7a92-459b-bae7-11dbfba90792 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:15.951461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45cf-7a92-459b-bae7-11dbfba90792 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:15.951461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45cf-7a92-459b-bae7-11dbfba90792 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:15.952807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 643/1288] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:15.957937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:15.958363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:15.958500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:15.977158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:15.978351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:16.007739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:16.044895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 646/1289] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:16.319290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b45cf-7a92-459b-bae7-11dbfba90792 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:16.418752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b45cf-7a92-459b-bae7-11dbfba90792 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:16.429874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:16.452506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:16.453125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:16.453404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:16.453676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:16.454250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:16.455648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:16.464204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:16.481468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:16.482711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 644/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:59:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:16.483646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:16.572041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c56b5-7936-47b4-acb6-f2af51d55197 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:16.606746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6460708-d736-469e-b686-aac906d4718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:16.607352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6460708-d736-469e-b686-aac906d4718e None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:16.607635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6460708-d736-469e-b686-aac906d4718e None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:16.607915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6460708-d736-469e-b686-aac906d4718e None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:16.608541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6460708-d736-469e-b686-aac906d4718e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:16.608841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6460708-d736-469e-b686-aac906d4718e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:16.610217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6460708-d736-469e-b686-aac906d4718e None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:16.615655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 647/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 05:59:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:16.621961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:16.719247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd572311-3fbd-4b1d-9221-cf84277ec6bd None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:16.751771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:16.752368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:16.752645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:16.752914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] PATH_INFO: `/v3/services` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:16.753537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:16.753834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:16.755208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ae6ab-661d-4c8f-90f7-6e5c7a2d4abb None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:16.759614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 645/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 05:59:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:18.662998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49d91-2356-4401-8ff0-53f4d9d73a1e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:18.663583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49d91-2356-4401-8ff0-53f4d9d73a1e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:18.663583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49d91-2356-4401-8ff0-53f4d9d73a1e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:18.664890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 648/1293] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:18.670287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32745bfa-40a3-467e-94c6-23429c34741b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:18.670738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32745bfa-40a3-467e-94c6-23429c34741b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:18.670864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32745bfa-40a3-467e-94c6-23429c34741b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:18.689041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-32745bfa-40a3-467e-94c6-23429c34741b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:18.690216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-32745bfa-40a3-467e-94c6-23429c34741b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:18.719517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:18.755163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 646/1294] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:19.026397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c49d91-2356-4401-8ff0-53f4d9d73a1e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:19.058709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:19.128839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c49d91-2356-4401-8ff0-53f4d9d73a1e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:19.163376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:19.163974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:19.164257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:19.164535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:19.165176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:19.166544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064d64c-ef07-4a75-87c1-3582120c0413 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:19.223830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 649/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Aug 26 05:59:19 2022] POST /identity/v3/services => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:59:21.122259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6b1b2-74ec-468f-8a03-75a804a95f45 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:21.122833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6b1b2-74ec-468f-8a03-75a804a95f45 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:21.122833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6b1b2-74ec-468f-8a03-75a804a95f45 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:21.124200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 647/1296] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:21.129481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:21.129900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:21.130039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:21.148137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:21.149271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:21.221537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 650/1297] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:21.498162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6b1b2-74ec-468f-8a03-75a804a95f45 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:21.602331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6b1b2-74ec-468f-8a03-75a804a95f45 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:21.636679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc499190-539f-452b-8b31-4988720f744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:21.637303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc499190-539f-452b-8b31-4988720f744e None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:21.637581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc499190-539f-452b-8b31-4988720f744e None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:21.637852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc499190-539f-452b-8b31-4988720f744e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:21.638429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc499190-539f-452b-8b31-4988720f744e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:21.639823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc499190-539f-452b-8b31-4988720f744e None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-bc499190-539f-452b-8b31-4988720f744e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:21.648487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:21.649654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:21.650497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 648/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:59:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:21.655600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:21.673843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:21.753170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31881be8-a8ec-4269-8afc-f0e3a2c3e172 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:21.786567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:21.787927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:21.788222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:21.788773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:21.789575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:21.790278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:21.791790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:21.796779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 651/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 05:59:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:21.803614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc499190-539f-452b-8b31-4988720f744e None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:21.834177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:21.899684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc499190-539f-452b-8b31-4988720f744e None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:21.934160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:21.934792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:21.935074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:21.935347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:21.936041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] RBAC: Adding query filter params (interface=public, service_id=549346240e784aa3b3e5df5810f6afec, region_id=RegionOne) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:21.936344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:21.937703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04f9fb-50dc-4be5-8a0f-1cc9ae18f3d9 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:21.942541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 649/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Aug 26 05:59:21 2022] GET /identity/v3/endpoints?service_id=549346240e784aa3b3e5df5810f6afec&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:21.948459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:22.029538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:22.047010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e1b8-fc0d-43db-8ea2-2e74f8ddb787 None admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:22.079854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:22.080450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:22.080731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:22.081003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:22.081602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:22.082989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b3812-b63f-4253-bb02-246c998deb77 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:22.090001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 652/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Aug 26 05:59:21 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 05:59:23.962690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340fd7c-32b9-4d30-8603-455419879c8a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:23.963237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340fd7c-32b9-4d30-8603-455419879c8a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:23.963237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340fd7c-32b9-4d30-8603-455419879c8a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:23.964539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 650/1302] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 05:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 05:59:23.969649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:23.972649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:23.972649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:23.988429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 05:59:23.989646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 05:59:24.058027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 653/1303] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 05:59:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 05:59:24.327758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340fd7c-32b9-4d30-8603-455419879c8a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:24.422895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:24.430177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340fd7c-32b9-4d30-8603-455419879c8a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:24.462902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:24.463530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:24.463809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:24.464571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:24.464674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:24.465998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 05:59:24.474560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 05:59:24.475872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application with Lock( Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self._enter() Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application return self.creator() Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application created_value = creator( Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: ERROR keystone.server.flask.application Aug 26 05:59:24.476896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 651/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Aug 26 05:59:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 05:59:24.481957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:24.560723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:24.582942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083d5e-c2d3-4688-9a4c-cc4f8dec4c8d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:24.615742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:24.616447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:24.616727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:24.617005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] PATH_INFO: `/v3/services` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:24.617644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 05:59:24.618138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:24.619782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28465c3-88a5-442c-9421-2147a57e5e66 None admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:24.625215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 654/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Aug 26 05:59:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 05:59:24.631224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 05:59:24.730150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e04a1-d8da-42c9-8898-6ca4159e10e0 None admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 05:59:24.763082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 05:59:24.763700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 05:59:24.763985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 05:59:24.764263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 05:59:24.764952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 05:59:24.766327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da50d5-2f4c-493b-a871-6cddcf9c8584 None admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 05:59:24.857897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 652/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 05:59:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 05:59:34.622192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=30 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 05:59:34.853385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:07:59.467888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335279c6-323c-460c-b053-f4e696939410 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:07:59.468356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335279c6-323c-460c-b053-f4e696939410 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:07:59.468356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335279c6-323c-460c-b053-f4e696939410 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:07:59.475001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b603df-9ad9-4aac-8631-152b827c69ef None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:07:59.475388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b603df-9ad9-4aac-8631-152b827c69ef None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:07:59.475499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b603df-9ad9-4aac-8631-152b827c69ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:07:59.478937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 655/1307] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:07:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:07:59.581086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f0b603df-9ad9-4aac-8631-152b827c69ef None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:07:59.582307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f0b603df-9ad9-4aac-8631-152b827c69ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:07:59.767331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 653/1308] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:08:00.059432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:00.059718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:00.060148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:00.061866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 656/1309] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 26 06:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:00.068191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966d2b-464a-4190-adc5-aa11ffcaca1a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:00.068763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966d2b-464a-4190-adc5-aa11ffcaca1a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:00.069153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966d2b-464a-4190-adc5-aa11ffcaca1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:00.159071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f7966d2b-464a-4190-adc5-aa11ffcaca1a None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:00.160319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f7966d2b-464a-4190-adc5-aa11ffcaca1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:00.185160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:00.247380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 654/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 26 06:08:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:00.259054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:00.260799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:00.381609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722267c3-5494-4d0b-ad62-bb8c926eb96b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:00.463053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:00.463669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:00.463950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:00.464223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:00.465821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:00.525232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:00.584669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:00.587222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5ed03-b86f-4286-8bc5-dc7b920c78c1 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:00.606912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 657/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:02.833167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:02.833738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:02.833738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:02.835822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 655/1312] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:02.840946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf365-99b0-41ff-9376-2eda635a7770 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:02.841347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf365-99b0-41ff-9376-2eda635a7770 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:02.841509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbf365-99b0-41ff-9376-2eda635a7770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:02.914068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-4ecbf365-99b0-41ff-9376-2eda635a7770 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:02.915607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4ecbf365-99b0-41ff-9376-2eda635a7770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:03.019506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 658/1313] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:03.302082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:03.303713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:03.406087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1535f8e-ca18-4f7e-ac26-7ab074d5a672 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:03.477253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:03.480051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:03.480682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:03.480958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:03.481223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:03.482850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:03.595723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:03.597161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b161f-36a3-4e6d-aaed-e8c38f6d5dba service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:03.618182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 656/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:06.205726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:06.206274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:06.206274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:06.207652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 659/1315] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:06.213339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedb97-277e-4f03-b903-6634bdf0ad7b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:06.213747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedb97-277e-4f03-b903-6634bdf0ad7b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:06.213893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedb97-277e-4f03-b903-6634bdf0ad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:06.278708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-37bedb97-277e-4f03-b903-6634bdf0ad7b None None] MFA Rules not processed for user `9a35490ab66448dc9fc3db978a411b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:06.279897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-37bedb97-277e-4f03-b903-6634bdf0ad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:06.329958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:06.373580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 657/1316] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:06.655752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:06.657323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:06.671111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:06.764989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c71cf-048d-4b60-9237-bc2bd6dd92eb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:06.838880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:06.839494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:06.839780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:06.840057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:06.841671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:06.903229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:06.950749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:06.952196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e76e2-2e52-4b37-84a5-d50a0e7ed105 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:06.970668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 660/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:09.700266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:09.700804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:09.700804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:09.702142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 658/1318] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:09.707313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae977c1e-04d5-4296-8823-5a2cccc69814 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:09.707697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae977c1e-04d5-4296-8823-5a2cccc69814 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:09.707869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae977c1e-04d5-4296-8823-5a2cccc69814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:09.772703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ae977c1e-04d5-4296-8823-5a2cccc69814 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:09.773903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ae977c1e-04d5-4296-8823-5a2cccc69814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:09.869518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 661/1319] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:10.153022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:10.154555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:10.257682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cfa0d-ad3f-44b5-b4c9-3fe29a53112d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:10.329469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:10.330072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:10.330354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:10.330651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:10.332266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:10.340450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:10.442181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:10.443649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c279d-ede5-42fe-9c1d-64d68491d099 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:10.462674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 659/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:13.382985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee771-f0d7-4651-9a13-34c658bb621d None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:13.383513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee771-f0d7-4651-9a13-34c658bb621d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:13.383513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dee771-f0d7-4651-9a13-34c658bb621d None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:13.384785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 662/1321] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:13.390015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7605d-7cda-4642-90a5-4ce6a893d18e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:13.390465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7605d-7cda-4642-90a5-4ce6a893d18e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:13.390581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7605d-7cda-4642-90a5-4ce6a893d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:13.458701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7ae7605d-7cda-4642-90a5-4ce6a893d18e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:13.460449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7ae7605d-7cda-4642-90a5-4ce6a893d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:13.519389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:13.565479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 660/1322] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:13.868712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:13.869136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:13.869507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:13.871040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 663/1323] 10.222.0.40 () {54 vars in 965 bytes} [Fri Aug 26 06:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:13.877623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff149a-99ab-41ea-bf08-ab9a5e02b48e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:13.878005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff149a-99ab-41ea-bf08-ab9a5e02b48e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:13.878354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff149a-99ab-41ea-bf08-ab9a5e02b48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:13.969929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-69ff149a-99ab-41ea-bf08-ab9a5e02b48e None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:13.971163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-69ff149a-99ab-41ea-bf08-ab9a5e02b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:14.051111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 661/1324] 10.222.0.40 () {58 vars in 1062 bytes} [Fri Aug 26 06:08:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:14.060842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:14.062519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:14.083240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:14.169575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ce3d6-34f0-4121-afea-1180a1a29529 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:14.245722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:14.246333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:14.246661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:14.246983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:14.248524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:14.320301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:14.359143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:14.360607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7997-508c-42bb-bab4-f8c582ed0b00 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:14.379063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 664/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:16.593824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:16.594350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:16.594350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:16.595887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 662/1326] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:16.601590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d07e-cc5e-4f81-9b05-43d23898cf34 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:16.602079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d07e-cc5e-4f81-9b05-43d23898cf34 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:16.602174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753d07e-cc5e-4f81-9b05-43d23898cf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:16.679461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b753d07e-cc5e-4f81-9b05-43d23898cf34 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:16.680732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b753d07e-cc5e-4f81-9b05-43d23898cf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:16.791464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 665/1327] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:08:17.130900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:17.133497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:17.198516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:17.244933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b260c40-edcc-4ca4-8193-84de795de310 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:17.321108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:17.321725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:17.322007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:17.322288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:17.323990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:17.435776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:17.438205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:17.439705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fa0b9-f7f7-4c9f-8218-8a93c91df927 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:17.458743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 663/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:19.515136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:19.515663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:19.515663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:19.517032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 666/1329] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:19.523311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a502e-e2f6-483f-9ad3-9d9d14133a49 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:19.524093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a502e-e2f6-483f-9ad3-9d9d14133a49 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:19.524093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a502e-e2f6-483f-9ad3-9d9d14133a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:19.596405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-df5a502e-e2f6-483f-9ad3-9d9d14133a49 None None] MFA Rules not processed for user `9a35490ab66448dc9fc3db978a411b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:19.597679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-df5a502e-e2f6-483f-9ad3-9d9d14133a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:19.698722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 664/1330] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:08:19.988447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:19.990002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:20.021691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:20.095166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b731c-362b-436f-ada2-4ee2f7840ff9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:20.176323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:20.176940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:20.177220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:20.177491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:20.179157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:20.261386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:20.297032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:20.298529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff04c7-bbc4-4cd9-8f2b-fabfc5abd931 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:20.318171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 667/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:19 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:22.686224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:22.686804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:22.686804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:22.688153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 665/1332] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:22.693365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cf33-abb6-47c8-aac2-997afe920871 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:22.693773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cf33-abb6-47c8-aac2-997afe920871 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:22.693980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6cf33-abb6-47c8-aac2-997afe920871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:22.761787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d0e6cf33-abb6-47c8-aac2-997afe920871 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:22.763033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d0e6cf33-abb6-47c8-aac2-997afe920871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:22.859700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 668/1333] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:23.146024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:23.148453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:23.219441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:23.254463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3562-6e73-47fc-a349-37c8417a7648 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:23.328726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:23.329261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:23.329543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:23.329819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:23.331456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:23.438691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:23.440173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17c058-3a63-4a85-ba22-bca36956ee8b service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:23.451202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:23.460831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 666/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:25.387342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:25.387876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:25.387876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:25.389230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 669/1335] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:25.394373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251a0e-b777-4f24-a5f4-0b4809630795 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:25.394824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251a0e-b777-4f24-a5f4-0b4809630795 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:25.394953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c251a0e-b777-4f24-a5f4-0b4809630795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:25.459376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8c251a0e-b777-4f24-a5f4-0b4809630795 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:25.461248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8c251a0e-b777-4f24-a5f4-0b4809630795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:25.562585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 667/1336] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:08:25.861931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:25.863546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:25.918694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:25.999865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37ca32-c797-4759-bf88-3ba22bf55b58 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:26.101554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:26.102146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:26.102425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:26.102727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:26.104380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:26.203560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:26.231358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:26.232835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf5ffc-5636-4df5-bda9-a67f7bc57551 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:26.252839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 670/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:28.926107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:28.926763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:28.926763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:28.929037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 668/1338] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:28.934465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df036da-b09a-4e03-918d-15199dd477dd None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:28.934975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df036da-b09a-4e03-918d-15199dd477dd None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:28.935128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df036da-b09a-4e03-918d-15199dd477dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:29.018401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7df036da-b09a-4e03-918d-15199dd477dd None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:29.020094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7df036da-b09a-4e03-918d-15199dd477dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:29.131542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 671/1339] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:29.424578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:29.426307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:29.505522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:29.534763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077087b0-ac45-4ab6-8a48-a33517a8bfb0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:29.621978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:29.622585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:29.622887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:29.623156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:29.624751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:29.746663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:29.748056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf318b-4f4e-4a98-ad9a-518b4a39c523 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:29.766525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 669/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:34.004024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:34.004664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:34.004664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:34.005940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 672/1341] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:34.013590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d18-ba70-4d69-a7d0-01257e2fa878 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:34.014136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d18-ba70-4d69-a7d0-01257e2fa878 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:34.014136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c0d18-ba70-4d69-a7d0-01257e2fa878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:34.019777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:34.090789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e59c0d18-ba70-4d69-a7d0-01257e2fa878 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:34.092011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e59c0d18-ba70-4d69-a7d0-01257e2fa878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:34.194022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 670/1342] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:34.494218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:34.495829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:34.549490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:34.606339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548e36-cd13-4623-88b3-96641e9db868 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:34.681093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:34.681706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:34.681984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:34.682253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:34.683881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:34.777289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:34.793554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:34.800597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcad42-b30b-4966-9074-da28f2515fe4 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:34.813378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 673/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:37.818184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:37.818897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:37.818897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:37.820112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 671/1344] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:37.825389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600bac2-1c26-407e-9a09-94d799f460e0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:37.825824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600bac2-1c26-407e-9a09-94d799f460e0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:37.826076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600bac2-1c26-407e-9a09-94d799f460e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:37.892847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1600bac2-1c26-407e-9a09-94d799f460e0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:37.894074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1600bac2-1c26-407e-9a09-94d799f460e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:37.993572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 674/1345] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:38.277922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:38.279501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:38.370608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:38.388677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbac0c-a6d3-41d8-ac5d-2a6c5b4d3ed5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:38.467990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:38.468613 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:38.468895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:38.469167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:38.470923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:38.582600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:38.584134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172e2c4-535f-44d4-ae19-efc318ce3f35 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:38.603119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 672/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:42.749097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:42.749629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:42.749629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:42.751103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 675/1347] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:42.756594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3313a-ca53-4276-b6b0-25a32fb7fd6f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:42.757114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3313a-ca53-4276-b6b0-25a32fb7fd6f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:42.757216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3313a-ca53-4276-b6b0-25a32fb7fd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:42.774333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:42.828934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-36d3313a-ca53-4276-b6b0-25a32fb7fd6f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:42.830150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-36d3313a-ca53-4276-b6b0-25a32fb7fd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:42.935156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 673/1348] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:43.221185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:43.222806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:43.285991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:43.337023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3040e-7985-4c4e-bca8-f41d79ba368e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:43.413658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:43.414267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:43.414553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:43.415113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:43.416747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:43.525787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:43.529187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:43.530589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53f9cf-af5a-48b1-bb61-d38e95069972 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:43.548536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 676/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:08:48.675611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ac435-b6ba-4d78-b900-66888600b4e5 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:48.676047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ac435-b6ba-4d78-b900-66888600b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:48.676254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ac435-b6ba-4d78-b900-66888600b4e5 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:48.677740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 674/1350] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:48.684070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b727f5b-17ce-499b-a053-f1b41a24183c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:48.684490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b727f5b-17ce-499b-a053-f1b41a24183c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:48.684663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b727f5b-17ce-499b-a053-f1b41a24183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:48.773550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-4b727f5b-17ce-499b-a053-f1b41a24183c None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:48.775087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4b727f5b-17ce-499b-a053-f1b41a24183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:48.971417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 677/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:08:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:52.933257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=59 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:53.493054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea4f82-ad75-479e-bc87-695cea6a71ff None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:53.493596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea4f82-ad75-479e-bc87-695cea6a71ff None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:53.493865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea4f82-ad75-479e-bc87-695cea6a71ff None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:53.495394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 675/1352] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:53.500668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34adbb-1177-41a3-871c-d7b5b27c9d60 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:53.501477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34adbb-1177-41a3-871c-d7b5b27c9d60 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:53.501939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34adbb-1177-41a3-871c-d7b5b27c9d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:53.562970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:53.588904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5b34adbb-1177-41a3-871c-d7b5b27c9d60 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:53.590262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5b34adbb-1177-41a3-871c-d7b5b27c9d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:53.720675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 678/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:08:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:08:54.853682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682eb5-fb70-4b87-a0ae-bc18cf412203 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:54.854180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682eb5-fb70-4b87-a0ae-bc18cf412203 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:54.854180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682eb5-fb70-4b87-a0ae-bc18cf412203 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:54.855483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 676/1354] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:54.860389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0f2df-9450-48fe-8b16-4b3f8fd8c030 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:54.860776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0f2df-9450-48fe-8b16-4b3f8fd8c030 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:54.860950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0f2df-9450-48fe-8b16-4b3f8fd8c030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:54.938434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b4b0f2df-9450-48fe-8b16-4b3f8fd8c030 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:54.939682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b4b0f2df-9450-48fe-8b16-4b3f8fd8c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:54.981829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:55.047237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 679/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:08:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:58.513757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a9efb-c94c-46a2-a470-363e41db28f4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:58.514299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a9efb-c94c-46a2-a470-363e41db28f4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:58.514299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a9efb-c94c-46a2-a470-363e41db28f4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:58.515975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 677/1356] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:58.521611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91143c-5b65-48b1-82ab-10f09aaebab2 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:58.522409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91143c-5b65-48b1-82ab-10f09aaebab2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:58.522754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91143c-5b65-48b1-82ab-10f09aaebab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:58.608845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9c91143c-5b65-48b1-82ab-10f09aaebab2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:58.610063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9c91143c-5b65-48b1-82ab-10f09aaebab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:58.761685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 680/1357] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:59.071400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:59.071939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:59.074499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:59.077374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 678/1358] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 06:08:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:08:59.085505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec840cd-18bd-4653-8079-afef900eca00 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:59.086057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec840cd-18bd-4653-8079-afef900eca00 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:59.087807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec840cd-18bd-4653-8079-afef900eca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:59.103554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:59.166646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1ec840cd-18bd-4653-8079-afef900eca00 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:08:59.168312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1ec840cd-18bd-4653-8079-afef900eca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:59.292034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 681/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 06:08:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:08:59.304787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:08:59.307616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:59.467485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f77e3-5714-421b-be78-31ce0f824115 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:08:59.547379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:08:59.591442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:08:59.592055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:08:59.592350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:08:59.592606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:08:59.594203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:08:59.686411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:08:59.688183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35894d2-085c-4066-b72d-37b0c2e54796 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:08:59.709397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 679/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:01.941601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:01.942098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:01.942563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:01.945112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 682/1361] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:01.953949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264fad-9a39-4702-a875-a751f0f54913 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:01.954478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264fad-9a39-4702-a875-a751f0f54913 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:01.954936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01264fad-9a39-4702-a875-a751f0f54913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:02.032080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-01264fad-9a39-4702-a875-a751f0f54913 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:02.033195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-01264fad-9a39-4702-a875-a751f0f54913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:02.085403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:02.157828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 680/1362] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:02.472504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:02.472504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:02.561516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:02.671465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563911d-cf02-4da8-8c16-9a5ace13b8c3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:02.814489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:02.815141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:02.815446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:02.816033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:02.818425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:02.853554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:02.948546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:02.950483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b59bb-ef61-4714-8d7e-607c2aeb60ac service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:02.972201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 683/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:05.607875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0bca2-18c2-43d3-8d3b-2544adeb3545 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:05.608396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0bca2-18c2-43d3-8d3b-2544adeb3545 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:05.608396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0bca2-18c2-43d3-8d3b-2544adeb3545 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:05.609664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 681/1364] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:05.615900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fb02-3266-416a-84d6-8ee5247caa4b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:05.616765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fb02-3266-416a-84d6-8ee5247caa4b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:05.617277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8fb02-3266-416a-84d6-8ee5247caa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:05.690391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d6a8fb02-3266-416a-84d6-8ee5247caa4b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:05.691833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d6a8fb02-3266-416a-84d6-8ee5247caa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:05.792264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:05.801289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 684/1365] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:06.088061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:06.088360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:06.088902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:06.090925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 682/1366] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 06:09:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:06.096887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305fa1-cc9a-4645-8e27-b36515e04e19 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:06.097253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305fa1-cc9a-4645-8e27-b36515e04e19 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:06.097608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305fa1-cc9a-4645-8e27-b36515e04e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:06.185271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-df305fa1-cc9a-4645-8e27-b36515e04e19 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:06.186503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-df305fa1-cc9a-4645-8e27-b36515e04e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:06.312862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 685/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 06:09:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:06.321570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:06.323375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:06.480249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e42196-2655-450f-a507-e63c1aee91f8 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:06.502957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:06.606044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:06.606676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:06.606961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:06.607231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:06.608908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:06.704207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:06.706078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6367d04-0562-4f67-88c9-b946578c05bb service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:06.726384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 683/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:07.033364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22eb6-5402-404e-8c94-115b0df013c2 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:07.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22eb6-5402-404e-8c94-115b0df013c2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:07.033883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab22eb6-5402-404e-8c94-115b0df013c2 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:07.035334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 686/1369] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:07.050892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119447-dc9c-4476-abea-c31f1d1eb0c2 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:07.050892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119447-dc9c-4476-abea-c31f1d1eb0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:07.050892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0119447-dc9c-4476-abea-c31f1d1eb0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:07.094101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:07.139585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f0119447-dc9c-4476-abea-c31f1d1eb0c2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:07.140868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f0119447-dc9c-4476-abea-c31f1d1eb0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:07.255753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 684/1370] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:07.855913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:07.855913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:07.855913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:07.858698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 687/1371] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 26 06:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:07.862171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388273a-b98f-4603-bdea-d6e6bc50601d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:07.862723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388273a-b98f-4603-bdea-d6e6bc50601d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:07.862874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388273a-b98f-4603-bdea-d6e6bc50601d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:07.956226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0388273a-b98f-4603-bdea-d6e6bc50601d None None] MFA Rules not processed for user `4710a6cf72564ff7abf8d35fa8e90a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:07.957503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0388273a-b98f-4603-bdea-d6e6bc50601d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:08.018269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:08.062171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 685/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 26 06:09:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:08.070542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:08.072144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:08.129193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:08.193011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3f3f-1c86-470e-b484-c1f26b6a2ffe None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:08.278367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:08.279006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:08.279292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:08.279566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:08.281182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:08.396104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:08.407238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:08.409083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4605876-2baf-4203-94c7-85771e364ff7 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:08.434978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 688/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:09.097172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:09.097718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:09.097718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:09.099194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 686/1374] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:09:09.104682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37242-9eff-48b7-b643-6e6b73fb0519 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:09.105209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37242-9eff-48b7-b643-6e6b73fb0519 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:09.105307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37242-9eff-48b7-b643-6e6b73fb0519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:09.175903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-58a37242-9eff-48b7-b643-6e6b73fb0519 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:09.177699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58a37242-9eff-48b7-b643-6e6b73fb0519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:09.280142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 689/1375] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:09.580702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:09.582279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:09.744931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48400432-8222-48bb-a4a3-aec29ba3a79e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:09.796839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:09.862373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:09.863154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:09.863432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:09.863702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:09.866176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:09.965174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:09.966851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8b6ba-0b02-442a-b05c-f7a0289c992d service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:09.986273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 687/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:12.187916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:12.188499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:12.188499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:12.189858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 690/1377] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:12.198711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2313-ed70-455a-9dd6-5c4042a19895 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:12.198711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2313-ed70-455a-9dd6-5c4042a19895 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:12.198711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2313-ed70-455a-9dd6-5c4042a19895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:12.291999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-579d2313-ed70-455a-9dd6-5c4042a19895 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:12.293460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-579d2313-ed70-455a-9dd6-5c4042a19895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:12.305165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:12.465133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 688/1378] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:12.756824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:12.758332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:12.824696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:12.920415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf01db-1c12-4210-918d-269d9c1b9bfa None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:13.042658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:13.043196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:13.043478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:13.043752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:13.045342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:13.065341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:13.151078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:13.152803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971eb61-c2eb-4ed9-b362-761be9a37c60 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:13.174270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 691/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:15.544677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:15.545538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:15.545975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:15.547548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 689/1380] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:15.554183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f019-e1d4-459a-9b1c-d1cd33528d31 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:15.554699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f019-e1d4-459a-9b1c-d1cd33528d31 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:15.554792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0f019-e1d4-459a-9b1c-d1cd33528d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:15.639605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-61e0f019-e1d4-459a-9b1c-d1cd33528d31 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:15.640737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-61e0f019-e1d4-459a-9b1c-d1cd33528d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:15.758203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:15.765343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 692/1381] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:09:16.091994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:16.093755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:16.257500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:16.271368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f397a4-33b4-4939-9658-c426051a2df0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:16.415339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:16.416895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:16.417298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:16.417652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:16.419501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:16.553181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:16.555247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d4b204-140e-47e2-96e0-bc937271454e service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:16.569785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:16.587670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 690/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:18.789540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:18.790720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:18.790720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:18.791835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 693/1383] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:09:18.797461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f001-a74f-4736-94fe-39579b58f9ff None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:18.798415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f001-a74f-4736-94fe-39579b58f9ff None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:18.798415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2f001-a74f-4736-94fe-39579b58f9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:18.870794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-91c2f001-a74f-4736-94fe-39579b58f9ff None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:18.872056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-91c2f001-a74f-4736-94fe-39579b58f9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:18.979945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 691/1384] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:09:19.269111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:19.271420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:19.459233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499d09a-9b12-4b05-9c0e-9faec6507842 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:19.534986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:19.584666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:19.585527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:19.585897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:19.586177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:19.588387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:19.706416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:19.709297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ecd6b-75e6-4331-9162-1e89480d45dc service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:19.733733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 694/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:21.703102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:21.703665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:21.703908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:21.706073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 692/1386] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:21.712071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc699-f6a7-438c-8411-cbcf83e8fb5c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:21.712517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc699-f6a7-438c-8411-cbcf83e8fb5c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:21.712666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc699-f6a7-438c-8411-cbcf83e8fb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:21.804055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5aebc699-f6a7-438c-8411-cbcf83e8fb5c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:21.805326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5aebc699-f6a7-438c-8411-cbcf83e8fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:21.840971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:21.909339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 695/1387] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:22.197992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:22.199541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:22.275532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:22.350860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1386-7976-4b41-8819-32ef42db7543 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:22.486665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:22.487818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:22.488371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:22.488822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:22.491692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:22.532695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:22.592332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:22.594120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffdddd4-69ce-4560-a4a6-f984fa80141f service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:22.621199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 693/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:24.437854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd525f-8c9f-4419-85df-34e1009fbde5 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.438454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd525f-8c9f-4419-85df-34e1009fbde5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.438454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd525f-8c9f-4419-85df-34e1009fbde5 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:24.439825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 696/1389] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:24.445735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a314-2ff0-4e45-b715-cc9db4545129 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.446163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a314-2ff0-4e45-b715-cc9db4545129 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.446264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4a314-2ff0-4e45-b715-cc9db4545129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:24.515537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-55d4a314-2ff0-4e45-b715-cc9db4545129 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:24.516728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-55d4a314-2ff0-4e45-b715-cc9db4545129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:24.624621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 694/1390] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:24.822903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae5b5-9b5f-49cf-b066-c4d5c12f4518 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.823218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae5b5-9b5f-49cf-b066-c4d5c12f4518 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.823575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ae5b5-9b5f-49cf-b066-c4d5c12f4518 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:24.825006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 697/1391] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:24.830367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922604-258e-433b-a10b-e42cb918f526 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.830681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922604-258e-433b-a10b-e42cb918f526 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.830962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74922604-258e-433b-a10b-e42cb918f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:24.846871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:24.903718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-74922604-258e-433b-a10b-e42cb918f526 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:24.904954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-74922604-258e-433b-a10b-e42cb918f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:24.929329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdea79a-bfba-40da-8751-9be623a93353 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.929671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdea79a-bfba-40da-8751-9be623a93353 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.929960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdea79a-bfba-40da-8751-9be623a93353 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:24.931689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 698/1392] 10.222.0.40 () {54 vars in 967 bytes} [Fri Aug 26 06:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:09:24.936810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:24.937260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:24.937521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:25.007387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 695/1393] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:25.019099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] MFA Rules not processed for user `4ee70da9d2f4451ba0d2133653b7d144`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:25.020719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:25.098920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 699/1394] 10.222.0.40 () {58 vars in 1064 bytes} [Fri Aug 26 06:09:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:09:25.106887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922604-258e-433b-a10b-e42cb918f526 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:25.108094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-74922604-258e-433b-a10b-e42cb918f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:25.206234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74922604-258e-433b-a10b-e42cb918f526 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:25.215121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:25.284530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:25.285134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:25.285407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:25.285680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:25.289651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:25.303107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:25.304340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:25.336398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:25.407830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:25.409235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90784bbc-1b3b-4728-8373-162729c3471b service glance] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:25.431641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 696/1395] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:25.432586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce718b89-8f5a-4db7-a0f4-ee89c7b20205 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:25.550209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:25.550961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:25.550961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:25.551179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:25.553486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:25.573685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:25.650861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:25.652886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fac6e0-ed4f-4701-a2fc-888fd3af7efe service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:25.674584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 700/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:09:26.421325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea124-68a8-4aec-8adb-f8b29ee1f0da None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:26.421797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea124-68a8-4aec-8adb-f8b29ee1f0da None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:26.421896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaea124-68a8-4aec-8adb-f8b29ee1f0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:26.467851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:26.490650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-eeaea124-68a8-4aec-8adb-f8b29ee1f0da None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:26.491911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eeaea124-68a8-4aec-8adb-f8b29ee1f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:26.581920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 697/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:26.624995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:26.625471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:26.625584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:26.626984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 701/1398] 10.222.0.40 () {54 vars in 943 bytes} [Fri Aug 26 06:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:26.635846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28aef-5314-4618-ab4a-f2444629fd0c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:26.636109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28aef-5314-4618-ab4a-f2444629fd0c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:26.636385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a28aef-5314-4618-ab4a-f2444629fd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:26.715433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f1a28aef-5314-4618-ab4a-f2444629fd0c None None] MFA Rules not processed for user `9d3c0f71e8984988b432390ec99977de`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:26.716634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f1a28aef-5314-4618-ab4a-f2444629fd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:26.779430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:26.825145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 698/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Fri Aug 26 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:26.835637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:26.837137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:26.979106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9ec5-765b-454d-a805-4e67d98683bb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:26.988442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:27.089019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:27.089633 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:27.090201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:27.090201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:27.091962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:27.178559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:27.180196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabada3-e876-49d9-aa22-db2968daf971 service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:27.188374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 702/1400] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:09:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:09:27.854670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:27.855179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:27.855179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:27.856531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 699/1401] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:27.861981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239141-346c-4259-92ca-afb0c69d29a0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:27.862439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239141-346c-4259-92ca-afb0c69d29a0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:27.862524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a239141-346c-4259-92ca-afb0c69d29a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:27.924822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:27.944974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7a239141-346c-4259-92ca-afb0c69d29a0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:27.946564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7a239141-346c-4259-92ca-afb0c69d29a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:28.068256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 703/1402] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:28.361409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:28.362990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:28.532523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc011fc3-ac1f-4a24-a222-94100aab106a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:28.579000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:28.676242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:28.677631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:28.678006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:28.678357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:28.680344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:28.788648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:28.790641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bee7e2-93e5-4889-b55a-6d5c46c88170 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:28.810429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 700/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:31.827446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:31.828038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:31.828038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:31.834152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 704/1404] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:31.836207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd4be-e0e7-49dd-897b-945788b3251e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:31.836700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd4be-e0e7-49dd-897b-945788b3251e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:31.836793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd4be-e0e7-49dd-897b-945788b3251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:31.930521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:31.943684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-110cd4be-e0e7-49dd-897b-945788b3251e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:31.947702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-110cd4be-e0e7-49dd-897b-945788b3251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:32.070101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 701/1405] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:32.365855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:32.371127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:32.496282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:32.532252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83238068-466d-4499-aa8a-38d60d8ad241 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:32.658244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:32.658860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:32.659145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:32.659418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:32.661721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:32.750274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:32.761531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:32.763175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39902943-f8c8-4511-a631-c6c94d40a983 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:32.782980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 705/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:09:34.984739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:34.985334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:34.985334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:34.986672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 702/1407] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:34.992102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f25c-52f6-4488-a387-e57354b28007 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:34.992891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f25c-52f6-4488-a387-e57354b28007 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:34.993176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f25c-52f6-4488-a387-e57354b28007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:35.077155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5dd1f25c-52f6-4488-a387-e57354b28007 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:35.078406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5dd1f25c-52f6-4488-a387-e57354b28007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:35.209152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 706/1408] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:35.526964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:35.528990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:35.669270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:35.699878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4e553-8121-4c68-8f4c-7a5cdef8103e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:35.829566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:35.830058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:35.830333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:35.830603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:35.832229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:35.939798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:35.944235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:35.945977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554dba1-ab9d-4133-96f4-2e304e106999 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:35.968922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 703/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:45.206638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=74 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:46.110800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:51.859136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26690a-2a41-46b8-9416-81409a2a4914 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:51.860053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26690a-2a41-46b8-9416-81409a2a4914 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:51.860053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26690a-2a41-46b8-9416-81409a2a4914 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:51.861415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 707/1410] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 26 06:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:09:51.868894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da947-c0b7-40d8-b659-4679a5a6d767 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:51.869455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da947-c0b7-40d8-b659-4679a5a6d767 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:51.869455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da947-c0b7-40d8-b659-4679a5a6d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:51.955008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-601da947-c0b7-40d8-b659-4679a5a6d767 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:51.956241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-601da947-c0b7-40d8-b659-4679a5a6d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:52.077685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 704/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 26 06:09:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:09:52.093509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19caa5-4d06-4422-8014-7df42c03f737 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:52.093802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19caa5-4d06-4422-8014-7df42c03f737 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:52.094088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19caa5-4d06-4422-8014-7df42c03f737 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:52.095674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 708/1412] 10.222.0.40 () {54 vars in 918 bytes} [Fri Aug 26 06:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:52.103117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f63-f1d2-4a9f-8243-00264b16f23a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:52.103543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f63-f1d2-4a9f-8243-00264b16f23a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:52.103703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c4f63-f1d2-4a9f-8243-00264b16f23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:52.182480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:52.185849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ec4c4f63-f1d2-4a9f-8243-00264b16f23a None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:52.187088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ec4c4f63-f1d2-4a9f-8243-00264b16f23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:52.327969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 705/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Aug 26 06:09:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:54.838131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:54.838676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:54.838676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:54.840028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:09:54.845737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f78305-c173-4ec6-86ac-114764afe7bd None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:54.846191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f78305-c173-4ec6-86ac-114764afe7bd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:54.846410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f78305-c173-4ec6-86ac-114764afe7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:54.918218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d4f78305-c173-4ec6-86ac-114764afe7bd None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:54.919553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d4f78305-c173-4ec6-86ac-114764afe7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:54.977801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:55.021697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:55.313910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:55.315504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:55.478845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d846e6b-067d-4337-8812-8d5cd06fae61 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:55.562997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:55.613538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:55.614370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:55.614686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:55.614972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:55.616703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:55.714479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:55.716145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1625756-89a0-45e5-b36f-3f02c1aaaf47 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:55.735835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 710/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:56.711663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:56.712234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:56.712234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:56.713664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:56.718752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eacc17-deac-4d12-95ca-89923a15f396 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:56.719300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eacc17-deac-4d12-95ca-89923a15f396 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:56.719300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eacc17-deac-4d12-95ca-89923a15f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:56.799279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-61eacc17-deac-4d12-95ca-89923a15f396 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:56.800416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-61eacc17-deac-4d12-95ca-89923a15f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:56.830446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:56.905719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:57.064396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:57.065891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:57.229484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830a4bc-6221-4e09-89b6-2a8e9899d90b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:57.274004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:57.353720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:57.354242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:57.354519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:57.354914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:57.356678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:57.457057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:57.458763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90398955-0a06-4979-ad33-524f5b4d17d6 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:57.477789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 708/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:09:58.373364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:58.373939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:58.373939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:58.375311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:09:58.380843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff2a3e-0cb7-4477-a1fe-3ce20ee3b5b3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:58.381270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff2a3e-0cb7-4477-a1fe-3ce20ee3b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:58.381384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff2a3e-0cb7-4477-a1fe-3ce20ee3b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:58.437268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:58.459809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f5ff2a3e-0cb7-4477-a1fe-3ce20ee3b5b3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:09:58.465209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f5ff2a3e-0cb7-4477-a1fe-3ce20ee3b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:58.565778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:09:58.726021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:09:58.751459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:58.889156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd6b07-f414-47eb-aaa9-63d5edc0fe03 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:09:58.904854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:09:59.018039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:09:59.018571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:09:59.018885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:09:59.019162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:09:59.020811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:09:59.121000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:09:59.122650 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8d211-733d-43d4-bc6e-8b6c830adc07 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:09:59.145422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 713/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:00.057629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:00.058164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:00.058164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:00.059554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:00.064845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6a48d-f707-4f5d-9c1f-2c939a7e06ac None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:00.065277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6a48d-f707-4f5d-9c1f-2c939a7e06ac None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:00.065380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6a48d-f707-4f5d-9c1f-2c939a7e06ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:00.087468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:00.145450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c6d6a48d-f707-4f5d-9c1f-2c939a7e06ac None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:00.146734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c6d6a48d-f707-4f5d-9c1f-2c939a7e06ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:00.246106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:00.400482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:00.402074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:00.519508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:00.560012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efd288-2924-46ca-9924-86eaba9ed339 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:00.715037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:00.715759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:00.716043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:00.716381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:00.718099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:00.807438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:00.821532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:00.823172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6a13a-792a-48b2-9ff3-c204ae0506d1 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:00.843901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 711/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:01.747049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:01.747634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:01.747634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:01.748954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:01.754575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079148c-b0fe-462d-8ab4-6a4dd90bdcb2 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:01.755070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079148c-b0fe-462d-8ab4-6a4dd90bdcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:01.755208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079148c-b0fe-462d-8ab4-6a4dd90bdcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:01.849206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5079148c-b0fe-462d-8ab4-6a4dd90bdcb2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:01.850432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5079148c-b0fe-462d-8ab4-6a4dd90bdcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:01.954116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:02.108871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:02.110320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:02.197602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:02.261739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68942e5-1d8c-4266-be4e-b0ced86f3a45 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:02.380150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:02.380673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:02.380973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:02.381253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:02.382916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:02.433658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:02.478974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:02.480674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4315c788-fe55-44c8-a74b-c3bf7e6601f1 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:02.499480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 716/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:03.510879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:03.511609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:03.511609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:03.512845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:03.518327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f853-524d-44e2-9886-30f05930a4b4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:03.520004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f853-524d-44e2-9886-30f05930a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:03.520004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97f853-524d-44e2-9886-30f05930a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:03.603909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2e97f853-524d-44e2-9886-30f05930a4b4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:03.605158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e97f853-524d-44e2-9886-30f05930a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:03.710590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:03.865306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:03.866799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:03.924330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:04.025064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cc03-2e50-4e42-84a5-7ac179bc1815 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:04.151208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:04.151779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:04.152063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:04.152328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:04.153925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:04.161179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:04.248302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:04.249924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623bdd-ac46-48e3-84ea-72122ff014c7 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:04.272319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 714/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:10:05.238591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:05.239164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:05.239164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:05.240523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:05.246124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200acd-9bf8-4c16-8bce-df22d74f2e67 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:05.246603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200acd-9bf8-4c16-8bce-df22d74f2e67 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:05.247116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97200acd-9bf8-4c16-8bce-df22d74f2e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:05.327321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-97200acd-9bf8-4c16-8bce-df22d74f2e67 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:05.329057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-97200acd-9bf8-4c16-8bce-df22d74f2e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:05.398364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:05.428450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:05.582423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:05.583912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:05.602421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:05.735016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0672ee-35b5-4230-ace2-585500fb0af1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:05.825163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:05.852174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:05.852735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:05.853521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:05.853521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:05.854879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:05.953297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:05.955042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c508ab4-737f-4747-80f5-12436b024489 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:05.974597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 719/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:07.057870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:07.058495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:07.058495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:07.059876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:07.064897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8157f7-b006-4ba5-94d2-8e6114a59255 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:07.065361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8157f7-b006-4ba5-94d2-8e6114a59255 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:07.065468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8157f7-b006-4ba5-94d2-8e6114a59255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:07.143565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0e8157f7-b006-4ba5-94d2-8e6114a59255 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:07.144940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0e8157f7-b006-4ba5-94d2-8e6114a59255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:07.189432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:07.247804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:07.401482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:07.403063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:07.554199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3061b-9786-4e41-a0c1-063fd0e25501 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:07.612831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:07.674377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:07.675045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:07.675393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:07.675886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:07.677593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:07.775668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:07.783633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f55a7f-6553-482b-9fef-371da271033f service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:07.803999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 717/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:08.716313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc521-d859-4171-aef6-b5128574e11d None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:08.716904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc521-d859-4171-aef6-b5128574e11d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:08.717018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dc521-d859-4171-aef6-b5128574e11d None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:08.718396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:08.725218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34412-d6c8-47e8-9b1a-3b10741445c5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:08.725930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34412-d6c8-47e8-9b1a-3b10741445c5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:08.725930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c34412-d6c8-47e8-9b1a-3b10741445c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:08.807964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f3c34412-d6c8-47e8-9b1a-3b10741445c5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:08.809191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f3c34412-d6c8-47e8-9b1a-3b10741445c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:08.811365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:08.913393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:09.067020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dc521-d859-4171-aef6-b5128574e11d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:09.068489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-775dc521-d859-4171-aef6-b5128574e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:09.218030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dc521-d859-4171-aef6-b5128574e11d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:09.247814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:09.345422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:09.345947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:09.346231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:09.346592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:09.350153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:09.463573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:09.465266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccdc5f-565f-4e50-9c97-1f15a88e7e1e service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:09.484213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 722/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:11.346370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:11.346989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:11.346989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:11.348956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:11.354873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41868-cb51-453e-8919-934ed2a75cbe None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:11.355379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41868-cb51-453e-8919-934ed2a75cbe None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:11.355482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a41868-cb51-453e-8919-934ed2a75cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:11.398152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:11.423954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-96a41868-cb51-453e-8919-934ed2a75cbe None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:11.425213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-96a41868-cb51-453e-8919-934ed2a75cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:11.529244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:11.837155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:11.839624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:11.961617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9223340-cdc6-48c8-84c9-b493cbaf5bd9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:11.997692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:12.044547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:12.045165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:12.045448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:12.045714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:12.047421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:12.166836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:12.168305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d3b01-d986-483f-a8e8-14fb453696c6 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:12.188396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 720/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:14.397304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:14.397914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:14.397914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:14.399307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:10:14.405117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7111-cd10-4b55-9c15-c698328e3858 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:14.405850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7111-cd10-4b55-9c15-c698328e3858 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:14.405850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e7111-cd10-4b55-9c15-c698328e3858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:14.485644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-094e7111-cd10-4b55-9c15-c698328e3858 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:14.486914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-094e7111-cd10-4b55-9c15-c698328e3858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:14.500944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:14.598728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:14.897866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:14.899673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:15.008180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfefef0-aeaa-4837-9c5d-038b2a56712f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:15.022914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:15.088285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:15.088927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:15.089511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:15.090212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:15.091927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:15.217500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:15.219076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704310cb-cd13-4761-886d-4c968c4013d5 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:15.244457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 725/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:18.013139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:18.018474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:18.018474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:18.018474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:18.021264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d08f-7c20-40a4-9e01-1c4878cdffc4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:18.021736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d08f-7c20-40a4-9e01-1c4878cdffc4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:18.021846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d08f-7c20-40a4-9e01-1c4878cdffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:18.193733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:18.212389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-51e2d08f-7c20-40a4-9e01-1c4878cdffc4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:18.213615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-51e2d08f-7c20-40a4-9e01-1c4878cdffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:18.313680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:18.601952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:18.603581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:18.710270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12a47-eecc-4378-9e3c-50fd3216bf1e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:18.749591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:18.782454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:18.783078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:18.783356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:18.783627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:18.785317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:18.895613 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:18.896990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929225b-b0dc-466f-a6e8-1b1edef6f147 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:18.916503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 723/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:20.988305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:20.988887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:20.988887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:20.990207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:20.995458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63efb-68e2-4cef-8cac-c9bc7c3015fc None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:20.995888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63efb-68e2-4cef-8cac-c9bc7c3015fc None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:20.996000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c63efb-68e2-4cef-8cac-c9bc7c3015fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:21.183328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b7c63efb-68e2-4cef-8cac-c9bc7c3015fc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:21.184625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b7c63efb-68e2-4cef-8cac-c9bc7c3015fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:21.206701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:21.284284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:21.570792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:21.572330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:21.697869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554729b4-79e5-4c1c-83a4-519fe07261fe None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:21.724398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:21.791425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:21.792038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:21.792317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:21.792590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:21.794216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:21.925735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:21.927619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325902-a965-4164-a9f3-e774630b4690 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:21.947950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 728/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:23.966847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:23.967379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:23.967379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:23.968802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:23.975067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e17c-b0cc-4ce5-9054-4a88235767cf None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:23.975489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e17c-b0cc-4ce5-9054-4a88235767cf None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:23.975600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366e17c-b0cc-4ce5-9054-4a88235767cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:24.038671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:24.041908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e366e17c-b0cc-4ce5-9054-4a88235767cf None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:24.044120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e366e17c-b0cc-4ce5-9054-4a88235767cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:24.142969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:24.432181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:24.433719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:24.539362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381cfbd-ff4f-411c-b9c1-5c56d714a66f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:24.591108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:24.616237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:24.616836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:24.617107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:24.617369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:24.619051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:24.742034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:24.743591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d9e7b-9773-41e4-9d7a-90e5fc5a02ef service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:24.763807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 726/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:10:26.753401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:26.753969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:26.753969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:26.755538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:26.760814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4457a-0559-4946-a13a-a767e9960e23 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:26.761254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4457a-0559-4946-a13a-a767e9960e23 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:26.761350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4457a-0559-4946-a13a-a767e9960e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:26.829083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-29f4457a-0559-4946-a13a-a767e9960e23 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:26.830294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-29f4457a-0559-4946-a13a-a767e9960e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:26.860631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:26.932615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:27.234030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:27.235619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:27.340937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020db2b-c390-428c-a5a0-ee8db21adb98 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:27.369563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:27.415621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:27.416232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:27.416515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:27.416796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:27.418414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:27.527238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:27.528690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d32289-4838-47b4-9311-a2fbdf223eea service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:27.548657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 731/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:29.621475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:29.622039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:29.622039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:29.623386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:29.629385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada936-7c5b-4639-a92a-4cbe5bfe2071 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:29.629784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada936-7c5b-4639-a92a-4cbe5bfe2071 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:29.629947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edada936-7c5b-4639-a92a-4cbe5bfe2071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:29.697848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-edada936-7c5b-4639-a92a-4cbe5bfe2071 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:29.699116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-edada936-7c5b-4639-a92a-4cbe5bfe2071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:29.708562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:29.796893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:30.092631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:30.094176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:30.197962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbab9b0-b08f-46dc-9b28-06df23d56046 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:30.252706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:30.271660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:30.275744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:30.275744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:30.275744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:30.275744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:30.384807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:30.386250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de706a3-5a48-40c3-b799-bfb143cbf0bc service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:30.405245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 729/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:32.372128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:32.372710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:32.372710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:32.374036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:32.380406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932dba-eb54-4679-8fc7-f085ac8ee54d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:32.380849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932dba-eb54-4679-8fc7-f085ac8ee54d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:32.380958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932dba-eb54-4679-8fc7-f085ac8ee54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:32.447009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b7932dba-eb54-4679-8fc7-f085ac8ee54d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:32.448231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b7932dba-eb54-4679-8fc7-f085ac8ee54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:32.487557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:32.545519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:32.828103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:32.829672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:32.935238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d2ea-1c1e-445c-872a-15d40f748fde None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:32.972768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:33.008207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:33.008947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:33.009536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:33.009813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:33.011448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:33.120663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:33.122073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524742a1-42b3-4d09-a100-981d16304cec service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:33.141576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 734/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:35.123122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:35.123706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:35.123706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:35.125275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:35.131758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f828ba-7dc9-48f1-ac2b-756dd7d5797f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:35.132292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f828ba-7dc9-48f1-ac2b-756dd7d5797f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:35.132446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f828ba-7dc9-48f1-ac2b-756dd7d5797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:35.204578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-36f828ba-7dc9-48f1-ac2b-756dd7d5797f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:35.205907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-36f828ba-7dc9-48f1-ac2b-756dd7d5797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:35.225718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:35.313336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:35.608028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:35.609552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:35.723935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021d36-4597-4064-96f4-fcc284645b4e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:35.797709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:35.805900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:35.807020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:35.807347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:35.807815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:35.809893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:35.926923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:35.928363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110216-7300-419b-924b-d32115b5351c service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:35.947025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 732/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:37.904124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:37.904644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:37.904747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:37.906024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:37.912372 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acee53-df06-4846-a17b-217d9d28dd83 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:37.912868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acee53-df06-4846-a17b-217d9d28dd83 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:37.912868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acee53-df06-4846-a17b-217d9d28dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:37.980940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f8acee53-df06-4846-a17b-217d9d28dd83 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:37.980940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f8acee53-df06-4846-a17b-217d9d28dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:38.031463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:38.083734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:38.380848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:10:38.382389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:38.488371 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b03be6-5d81-4cca-b58a-8c3be034cc33 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:10:38.536868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:38.563741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:10:38.564350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:38.564634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:38.564912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:38.566539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:38.675997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:10:38.677449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe2aef-0ff4-4fc2-90e4-5cb23b7a2c8b service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:10:38.695902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 737/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:10:48.102849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=22 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:48.693580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=66 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:10:59.438730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8bde1-354a-44c6-830f-bbb46d5dd325 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:59.439338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8bde1-354a-44c6-830f-bbb46d5dd325 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:59.439338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8bde1-354a-44c6-830f-bbb46d5dd325 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:59.441316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 734/1471] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:59.449874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db26c4-99f8-4c4f-944c-6886fbc43a58 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:59.450456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db26c4-99f8-4c4f-944c-6886fbc43a58 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:59.450456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db26c4-99f8-4c4f-944c-6886fbc43a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:59.528707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b3db26c4-99f8-4c4f-944c-6886fbc43a58 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:10:59.529929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b3db26c4-99f8-4c4f-944c-6886fbc43a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:10:59.639331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 738/1472] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:10:59.942292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:59.942578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:59.942888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:10:59.944246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 735/1473] 10.222.0.40 () {54 vars in 967 bytes} [Fri Aug 26 06:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:10:59.949139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0566-9066-4e74-9db2-b2e6e425a141 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:10:59.949431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0566-9066-4e74-9db2-b2e6e425a141 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:10:59.949723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0566-9066-4e74-9db2-b2e6e425a141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:00.031901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-180f0566-9066-4e74-9db2-b2e6e425a141 None None] MFA Rules not processed for user `4ee70da9d2f4451ba0d2133653b7d144`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:00.034276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-180f0566-9066-4e74-9db2-b2e6e425a141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:00.056895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:00.117158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 739/1474] 10.222.0.40 () {58 vars in 1064 bytes} [Fri Aug 26 06:10:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:11:00.125248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:00.127500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:00.237360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0c243-add2-45b0-97d9-273f23f3236f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:00.315803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:00.316449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:00.316731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:00.317005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:00.318681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:00.376088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:00.433488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:00.434972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9da35-66f7-4b0d-80ff-7901856dbce4 service glance] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:00.456481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 736/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:11:02.560948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:02.561499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:02.561499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:02.562900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 740/1476] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:11:02.568851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486af74-22c9-474c-b601-115d360c6c10 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:02.569347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486af74-22c9-474c-b601-115d360c6c10 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:02.569451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486af74-22c9-474c-b601-115d360c6c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:02.648282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9486af74-22c9-474c-b601-115d360c6c10 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:02.649516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9486af74-22c9-474c-b601-115d360c6c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:02.751780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 737/1477] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:03.026311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:03.027915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:03.160736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90a97-428d-4886-9492-073d2a5b1f63 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:03.212033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:03.265192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:03.266053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:03.266809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:03.267202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:03.268057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:03.270289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:11:03.270289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:11:03.270289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:11:03.271772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17d37-a549-47d2-8325-418f9bccefa2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:03.293749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 741/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Aug 26 06:11:03 2022] GET /identity/v3/projects => generated 2553 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:11:05.289348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465075-7436-4956-9355-b29baca56eb1 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:05.289923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465075-7436-4956-9355-b29baca56eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:05.289923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465075-7436-4956-9355-b29baca56eb1 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:05.291547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 738/1479] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:11:05.297127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a2697-44b3-42e2-b405-5b806ec575ee None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:05.297563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a2697-44b3-42e2-b405-5b806ec575ee None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:05.297686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a2697-44b3-42e2-b405-5b806ec575ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:05.366753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-865a2697-44b3-42e2-b405-5b806ec575ee None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:05.368000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-865a2697-44b3-42e2-b405-5b806ec575ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:05.466525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 742/1480] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:05.765407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49465075-7436-4956-9355-b29baca56eb1 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:05.767012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-49465075-7436-4956-9355-b29baca56eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:05.782824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:05.927201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49465075-7436-4956-9355-b29baca56eb1 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:06.013399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:06.043563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:06.044195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:06.044479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:06.044762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:06.046483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:06.144409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:06.147175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac6ac9-89db-49b7-828d-4a7b03e81703 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:06.167839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 739/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:11:08.328487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:08.329163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:08.329163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:08.330798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 743/1482] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:11:08.336123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e105d4-688b-4590-8222-a39d5425353d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:08.336716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e105d4-688b-4590-8222-a39d5425353d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:08.337002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e105d4-688b-4590-8222-a39d5425353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:08.404984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-36e105d4-688b-4590-8222-a39d5425353d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:08.406206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-36e105d4-688b-4590-8222-a39d5425353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:08.454893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:08.506663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 740/1483] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:08.794232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:08.795881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:08.820017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:08.901987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b8751-2ae5-4b61-abbd-39e6badb6724 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:08.987309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:08.988049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:08.988526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:08.988810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:08.990498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:09.069817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:09.105459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:09.106952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ed86b-e6d6-471d-bc77-71d19329d949 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:09.127052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 744/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:11:11.157873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:11.158352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:11.158457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:11.159867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 741/1485] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:11:11.166481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d98e-4b60-4377-8bed-5996ea3757de None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:11.167030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d98e-4b60-4377-8bed-5996ea3757de None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:11.167139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6d98e-4b60-4377-8bed-5996ea3757de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:11.239136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-86a6d98e-4b60-4377-8bed-5996ea3757de None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:11.240459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-86a6d98e-4b60-4377-8bed-5996ea3757de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:11.345843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 745/1486] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:11.647233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:11.648786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:11.757281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760edb2-46a1-492e-ab4c-c067375a7037 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:11.838371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:11.840223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:11.840833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:11.841109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:11.841380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:11.843057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:11.964754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:11.966196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146565e8-f8d6-476a-8abd-984e5ab45dcf service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:11.985473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 742/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:11:14.020242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:14.020822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:14.020978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:14.022402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 746/1488] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:11:14.028167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b53c-265e-46e1-b18d-4e76ad885be3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:14.028653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b53c-265e-46e1-b18d-4e76ad885be3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:14.028765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7b53c-265e-46e1-b18d-4e76ad885be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:14.098181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7dd7b53c-265e-46e1-b18d-4e76ad885be3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:14.099439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7dd7b53c-265e-46e1-b18d-4e76ad885be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:14.160491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:14.210454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 743/1489] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:14.504131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:14.505746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:14.544302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:14.617426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf067a9-999d-46ac-863e-d3da845e90cc None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:14.705562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:14.706724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:14.707116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:14.707402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:14.709097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:14.795514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:14.827335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:14.828804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f01463-1698-477d-b64b-7a679324dd81 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:14.848447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 747/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:11:17.350325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:17.350935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:17.351044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:17.352425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 744/1491] 10.222.0.40 () {54 vars in 927 bytes} [Fri Aug 26 06:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:11:17.359517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d098b-e070-4f16-b48a-747de15a07b8 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:17.360076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d098b-e070-4f16-b48a-747de15a07b8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:17.360076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d098b-e070-4f16-b48a-747de15a07b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:17.435043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6f9d098b-e070-4f16-b48a-747de15a07b8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:11:17.436306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6f9d098b-e070-4f16-b48a-747de15a07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:17.553664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 748/1492] 10.222.0.40 () {58 vars in 1024 bytes} [Fri Aug 26 06:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:11:17.855774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:11:17.857319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:17.994183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa63935-fe4f-4783-a23e-99527b0cf514 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:11:18.085349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:11:18.086086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:11:18.086370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:11:18.086736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:11:18.088516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:11:18.099193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:18.222204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:11:18.223793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b3ce4-4ff7-4201-b86e-80430c0c09a1 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:11:18.245071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 745/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:11:27.551292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=82 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:11:28.242635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=50 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:44.409463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:44.410069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:44.410069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:44.489155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:12:44.490376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:44.588268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 749/1494] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:12:44.596869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:44.597378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:44.597485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:44.675393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:12:44.676530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:44.772315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 746/1495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:12:44.777015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:44.778237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:44.790731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a298f-f0a0-48fa-9b00-77902a0a5984 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:44.852465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:44.887377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:44.887988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:44.888269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:44.888543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:44.890040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:12:44.890603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:44.891961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:44.891961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:12:44.891961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:12:44.893178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:44.906521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 750/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:12:44.914916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:44.924713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ff0b7-2d0b-4d0f-88c2-22c919b03b84 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:44.986537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:45.020073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:45.020682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:45.020953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:45.021222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:45.021850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:45.023616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:45.023616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:12:45.023616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:12:45.025414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:45.169155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 747/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:12:44 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:12:45.173661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:45.175307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:45.210069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a0bea-8135-4b82-b566-e17c4d28ab87 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:45.305151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:45.305675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:45.305954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:45.306228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:45.306892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:45.307915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:45.307915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:12:45.307915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:12:45.309113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:45.430547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 751/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:12:45 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:12:45.434747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:45.436993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:45.500144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685d60c-7424-4042-8fce-a150ad1acfd1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:45.584772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:45.596335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:12:45.600883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:12:45.602069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:45.613413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 748/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:12:45.617693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:45.619401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:45.631294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ee92-9222-423a-b701-9d5f7b10e5cd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:45.657726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:45.727506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:45.728116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:45.728397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:45.728670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:45.729316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:45.730374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:45.730374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:12:45.730374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:12:45.731317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3d1b-1fe8-44ff-b28a-ab57e0ce793a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:45.747788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 752/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:12:45.752258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:45.753885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:45.789248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf1597-3695-4a17-91a8-f04cf312f55f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:45.882787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:45.883364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:45.883643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:45.883915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] PATH_INFO: `/v3/projects/ba2cc9e9a5ca40ce978dd3fa536301ac/users/b562779fb87741899645fc59efe88e2a/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:45.907380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2cc9e9a5ca40ce978dd3fa536301ac, user_id=b562779fb87741899645fc59efe88e2a, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:45.908428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:45.908428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:12:45.908428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:12:45.910389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f0fe3-71df-46d0-a0a5-35b9342e64f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:46.191332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 749/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:12:45 2022] PUT /identity/v3/projects/ba2cc9e9a5ca40ce978dd3fa536301ac/users/b562779fb87741899645fc59efe88e2a/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:12:46.204900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2ce4-2faa-42be-8cb3-c2e3d5b970a5 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:46.205188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2ce4-2faa-42be-8cb3-c2e3d5b970a5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:46.205508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb2ce4-2faa-42be-8cb3-c2e3d5b970a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:46.263333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e0eb2ce4-2faa-42be-8cb3-c2e3d5b970a5 None None] MFA Rules not processed for user `b562779fb87741899645fc59efe88e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:12:46.265119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e0eb2ce4-2faa-42be-8cb3-c2e3d5b970a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:46.343856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 753/1502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:12:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:12:46.356687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:46.356978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:46.357258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:46.358664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 750/1503] 10.222.0.40 () {54 vars in 964 bytes} [Fri Aug 26 06:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:12:46.363339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e70b3-84a0-46f9-99af-89423fe44e17 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:46.363628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e70b3-84a0-46f9-99af-89423fe44e17 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:46.363929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e70b3-84a0-46f9-99af-89423fe44e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:46.396519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:46.441835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-204e70b3-84a0-46f9-99af-89423fe44e17 None None] MFA Rules not processed for user `4710a6cf72564ff7abf8d35fa8e90a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:12:46.442985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-204e70b3-84a0-46f9-99af-89423fe44e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:46.521287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 754/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Aug 26 06:12:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:12:46.528948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:46.530528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:46.583747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:46.638327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f490afc-2362-4ffe-aaa8-8d7bfa33e5e5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:46.713432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:46.714009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:46.714287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:46.714574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:46.716230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:46.817699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:46.819171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:46.822649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:46.838485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 751/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:12:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:12:46.883765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479611-a03d-4f1a-bce4-aec85440fa70 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:46.884061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479611-a03d-4f1a-bce4-aec85440fa70 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:46.884346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46479611-a03d-4f1a-bce4-aec85440fa70 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:46.885652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 755/1506] 10.222.0.40 () {52 vars in 830 bytes} [Fri Aug 26 06:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:12:46.889913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:46.891186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:46.991460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35576020-d914-4989-bc79-c8ddf7dc5c51 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:47.084842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:47.085464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:47.085740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:47.086005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] PATH_INFO: `/v3/users/b562779fb87741899645fc59efe88e2a` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:47.098873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b562779fb87741899645fc59efe88e2a)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:47.099894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:47.099894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:12:47.099894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:12:47.101096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:47.111097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:47.375960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b562779fb87741899645fc59efe88e2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:47.376324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b562779fb87741899645fc59efe88e2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:47.399879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b562779fb87741899645fc59efe88e2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:47.480681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b562779fb87741899645fc59efe88e2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:47.495360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b562779fb87741899645fc59efe88e2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:47.502103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 752/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:12:46 2022] DELETE /identity/v3/users/b562779fb87741899645fc59efe88e2a => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:12:47.506436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:47.506951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:47.507047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:47.564908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:12:47.566112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:47.641897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:47.654768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 756/1508] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:12:47.665026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:47.666497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:47.769500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d884919-6ad1-4187-90ab-f39e40c630b2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:47.824127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:47.844409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:47.845018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:47.845298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:47.845615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:47.847270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:47.957233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:47.958703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac7ac-c731-422f-af38-de1c06ae2008 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:47.977336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 753/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:12:48.535552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:12:48.544316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:12:48.549746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1986bd-c63f-423a-b6b9-86fb8f2e8ea2 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:12:48.653947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:12:48.655373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:12:48.655659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:12:48.655936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] PATH_INFO: `/v3/projects/ba2cc9e9a5ca40ce978dd3fa536301ac` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:12:48.666265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2cc9e9a5ca40ce978dd3fa536301ac)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:12:48.667390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:12:48.667390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:12:48.667390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:12:48.668577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:12:48.816268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2cc9e9a5ca40ce978dd3fa536301ac', 'request_id': 'req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:48.822713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2cc9e9a5ca40ce978dd3fa536301ac', 'request_id': 'req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:48.926733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2cc9e9a5ca40ce978dd3fa536301ac', 'request_id': 'req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:48.959043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] The token cache is being invalidate because project ba2cc9e9a5ca40ce978dd3fa536301ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:12:48.959362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:48.959852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a9e847fa-1d8d-4171-a2ba-86c0f40f9cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:12:48.961327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 757/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:12:48 2022] DELETE /identity/v3/projects/ba2cc9e9a5ca40ce978dd3fa536301ac => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:12:57.975038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=62 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:12:58.956700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:14:24.372728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:24.373357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:24.373357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:24.374608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:24.379640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e53f-46ee-4461-af69-b85f9f222d73 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:24.380105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e53f-46ee-4461-af69-b85f9f222d73 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:24.380231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728e53f-46ee-4461-af69-b85f9f222d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:24.463119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c728e53f-46ee-4461-af69-b85f9f222d73 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:24.465106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c728e53f-46ee-4461-af69-b85f9f222d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:24.575377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:24.730467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:24.732635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:24.854549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ee5a-821e-4e12-a445-d778e36d71c6 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:24.860149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:24.860760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:24.861044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:24.861321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:24.869678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:24.871647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c94b-81c7-4841-b53c-fd970232884e service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:24.881157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 755/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:25.717750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557715-0d18-4580-b063-8f89d05dc14a None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:25.718343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557715-0d18-4580-b063-8f89d05dc14a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:25.718343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557715-0d18-4580-b063-8f89d05dc14a None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:25.719684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:25.724523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4849a-328c-4b0d-8e3c-11ec56bb7cdf None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:25.724936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4849a-328c-4b0d-8e3c-11ec56bb7cdf None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:25.725084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4849a-328c-4b0d-8e3c-11ec56bb7cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:25.745355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-dfe4849a-328c-4b0d-8e3c-11ec56bb7cdf None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:25.747161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-dfe4849a-328c-4b0d-8e3c-11ec56bb7cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:25.754638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:25.908254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6557715-0d18-4580-b063-8f89d05dc14a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:25.924204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6557715-0d18-4580-b063-8f89d05dc14a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:25.929954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:25.930594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:25.930906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:25.931208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:25.940091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:25.941712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e02e0-f79f-4cbe-b3dc-c96b39851875 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:25.951850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 760/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:26.353895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f332a-d59d-407d-8ee8-8cfcd8138518 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:26.354177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f332a-d59d-407d-8ee8-8cfcd8138518 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:26.354471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f332a-d59d-407d-8ee8-8cfcd8138518 None None] PATH_INFO: `/v3` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:26.355946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 757/1517] 10.222.0.76 () {54 vars in 925 bytes} [Fri Aug 26 06:14:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:14:26.361986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932cbc-60bc-40e6-8082-af8cbd5d5c53 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:26.362272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932cbc-60bc-40e6-8082-af8cbd5d5c53 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:26.362548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59932cbc-60bc-40e6-8082-af8cbd5d5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:26.424134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-59932cbc-60bc-40e6-8082-af8cbd5d5c53 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:26.425350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-59932cbc-60bc-40e6-8082-af8cbd5d5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:26.442275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:14:26.498740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 761/1518] 10.222.0.76 () {58 vars in 1015 bytes} [Fri Aug 26 06:14:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:30.620869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc02d-420f-4a03-af20-48791489695c None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:30.621506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc02d-420f-4a03-af20-48791489695c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:30.621506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecc02d-420f-4a03-af20-48791489695c None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:30.622927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 758/1519] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 26 06:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:30.628247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb121f0-9872-4474-9d3d-7fa11af88ca7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:30.628707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb121f0-9872-4474-9d3d-7fa11af88ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:30.628707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb121f0-9872-4474-9d3d-7fa11af88ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:30.648645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8bb121f0-9872-4474-9d3d-7fa11af88ca7 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:30.649880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8bb121f0-9872-4474-9d3d-7fa11af88ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:30.659191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 762/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 26 06:14:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:30.667523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecc02d-420f-4a03-af20-48791489695c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:30.680870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecc02d-420f-4a03-af20-48791489695c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:30.686454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:30.687075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:30.687356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:30.687631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:30.696151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:30.698893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f450db-6675-45c2-8ed8-1dd948d8a8fd service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:30.708090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 759/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:37.202530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d617e-3633-4b19-9bf9-976264fd9fa6 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:37.203423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d617e-3633-4b19-9bf9-976264fd9fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:37.203586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d617e-3633-4b19-9bf9-976264fd9fa6 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:37.204991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:14:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:37.210536 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040316b-2047-4128-b20e-2082ceb524c0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:37.211153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040316b-2047-4128-b20e-2082ceb524c0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:37.211153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040316b-2047-4128-b20e-2082ceb524c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:37.232328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1040316b-2047-4128-b20e-2082ceb524c0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:37.233547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1040316b-2047-4128-b20e-2082ceb524c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:37.241886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:37.411116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d617e-3633-4b19-9bf9-976264fd9fa6 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:37.427083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d617e-3633-4b19-9bf9-976264fd9fa6 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:37.433418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:37.434025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:37.434307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:37.434582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:37.442897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:37.444591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143051ce-31e2-49a8-8851-e3b88578bff9 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:37.455326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 764/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:38.294101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300327-1b30-4ab5-9eb6-71cc21c506ed None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:38.294686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300327-1b30-4ab5-9eb6-71cc21c506ed None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:38.294686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300327-1b30-4ab5-9eb6-71cc21c506ed None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:38.296070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 06:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:38.301178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b88839-d81a-4c61-872c-c91c413f3c45 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:38.301637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b88839-d81a-4c61-872c-c91c413f3c45 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:38.301738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b88839-d81a-4c61-872c-c91c413f3c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:38.321044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-72b88839-d81a-4c61-872c-c91c413f3c45 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:38.322282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-72b88839-d81a-4c61-872c-c91c413f3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:38.331189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 06:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:38.486853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50300327-1b30-4ab5-9eb6-71cc21c506ed None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:38.504226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50300327-1b30-4ab5-9eb6-71cc21c506ed None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:38.511576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:38.512236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:38.512517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:38.512868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:38.521966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:38.523908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03cd54-b027-4167-819e-c0963398eb6c service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:38.535570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 762/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:42.972518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:42.973082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:42.973082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:42.974419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:42.981095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734c762-4541-4acb-b0b6-e2cce86d91a4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:42.981599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734c762-4541-4acb-b0b6-e2cce86d91a4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:42.981747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734c762-4541-4acb-b0b6-e2cce86d91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:43.002439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0734c762-4541-4acb-b0b6-e2cce86d91a4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:43.003682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0734c762-4541-4acb-b0b6-e2cce86d91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:43.011722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:43.308991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:43.311102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:43.394796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f8e0a-98f0-4402-8323-2d4f86864f8d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:43.399752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:43.400446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:43.400777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:43.401093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:43.410826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:43.412279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01423e-fb00-4331-946c-1f0cfc1cc869 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:43.423091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 767/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:45.393850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ddffb-2115-4d40-9eaf-c9bcbf982a61 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:45.394538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ddffb-2115-4d40-9eaf-c9bcbf982a61 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:45.394538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ddffb-2115-4d40-9eaf-c9bcbf982a61 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:45.395817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:45.402061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b304b-0cc5-43ca-b198-678988845421 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:45.402829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b304b-0cc5-43ca-b198-678988845421 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:45.402829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b304b-0cc5-43ca-b198-678988845421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:45.422328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a28b304b-0cc5-43ca-b198-678988845421 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:45.423586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a28b304b-0cc5-43ca-b198-678988845421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:45.432075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:45.720969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ddffb-2115-4d40-9eaf-c9bcbf982a61 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:45.732623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ddffb-2115-4d40-9eaf-c9bcbf982a61 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:45.736077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:45.736664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:45.736940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:45.737209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:45.747188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:45.748617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ef6c6-18f1-4d84-8eff-ff56981750b6 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:45.758212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 765/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:49.758528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:49.759307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:49.759570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:49.761130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:49.766755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a8f6-2d58-44a0-acb3-52a88c3df307 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:49.767234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a8f6-2d58-44a0-acb3-52a88c3df307 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:49.767338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0a8f6-2d58-44a0-acb3-52a88c3df307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:49.775708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=81 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:14:49.788228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-18e0a8f6-2d58-44a0-acb3-52a88c3df307 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:49.789458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-18e0a8f6-2d58-44a0-acb3-52a88c3df307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:49.798332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:50.372889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:50.374697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:50.446703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:14:50.462322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4250f1a-8ed1-4443-a3e6-1aa2edd75e0e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:50.469557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:50.469557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:50.469557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:50.469557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:50.477965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:50.479557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef5559-93ae-44a7-aae0-e7f1ed74c9ed service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:50.489935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 770/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:14:52.524867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:52.525447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:52.525447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:52.526882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 06:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:14:52.532701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbedb-e626-4b42-9ba3-bde4176084e1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:52.533291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbedb-e626-4b42-9ba3-bde4176084e1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:52.533291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbedb-e626-4b42-9ba3-bde4176084e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:52.555296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2b6cbedb-e626-4b42-9ba3-bde4176084e1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:14:52.556956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2b6cbedb-e626-4b42-9ba3-bde4176084e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:52.565213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 06:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:14:53.124795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:14:53.126554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:14:53.140208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a1d3f-8ead-4238-a5eb-a97868c6946d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:14:53.143759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:14:53.144356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:14:53.144632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:14:53.144910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:14:53.154417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:14:53.155820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3e920-1fc5-4146-928c-2240c8a726de service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:14:53.166064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 768/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:15:02.548536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=11 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:15:03.161067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=8 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:19.267943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.268681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.268681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.299207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.300504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.312175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 772/1540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:19.322627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.323252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.323252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.358786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.360028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.377660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 769/1541] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.382495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:19.403036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7cefd7-1de9-4318-9d50-980394f51b94 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:19.411853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:19.413313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.413610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.413891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.414630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:19.414970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:19.417752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842aed4-11cf-4d34-bef0-43b783f9bf1b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:19.444545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 773/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:19.453074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:19.468120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e8069-2e26-4338-903d-8f65e76ba211 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:19.473296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:19.473984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.474241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.474502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.475864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:19.477869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:19.606793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9131e-b931-4f8b-97c8-212eb0cff641 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.607505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9131e-b931-4f8b-97c8-212eb0cff641 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.608005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9131e-b931-4f8b-97c8-212eb0cff641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.609878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 770/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:19.613654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:19.645308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c4f9131e-b931-4f8b-97c8-212eb0cff641 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.656137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c4f9131e-b931-4f8b-97c8-212eb0cff641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.737273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745508-6058-4e19-97ae-7c96b0144ce2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:19.746703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:19.747455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.747739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.748057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.748763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:19.749917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:19.749917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:16:19.749917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:16:19.751169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01b1e-3365-44bd-9050-f2c7861f03d9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:19.754781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 774/1544] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.756340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7062-0d85-47c5-afe3-653fddd8b58c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.756858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7062-0d85-47c5-afe3-653fddd8b58c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.757370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7062-0d85-47c5-afe3-653fddd8b58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.794952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-502a7062-0d85-47c5-afe3-653fddd8b58c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.799447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-502a7062-0d85-47c5-afe3-653fddd8b58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.809302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 775/1545] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.811165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d042-bbdc-4eb9-88b1-f758750c23de None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.811710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d042-bbdc-4eb9-88b1-f758750c23de None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.812229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82d042-bbdc-4eb9-88b1-f758750c23de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.845642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2a82d042-bbdc-4eb9-88b1-f758750c23de None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.847709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2a82d042-bbdc-4eb9-88b1-f758750c23de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.857846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 776/1546] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.859479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74302-9a32-4ad5-9980-bfaf6b4c8d55 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.859788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74302-9a32-4ad5-9980-bfaf6b4c8d55 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.860068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d74302-9a32-4ad5-9980-bfaf6b4c8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.884494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a1d74302-9a32-4ad5-9980-bfaf6b4c8d55 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.885727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a1d74302-9a32-4ad5-9980-bfaf6b4c8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.894266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 777/1547] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.895563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6d6e-7c38-4374-aeff-d568e68b3ecf None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.895858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6d6e-7c38-4374-aeff-d568e68b3ecf None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.896165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6d6e-7c38-4374-aeff-d568e68b3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.905763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 771/1548] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:16:19.907514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872e90c-25e3-4add-ab77-08100cdee76b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.907809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872e90c-25e3-4add-ab77-08100cdee76b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.908086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872e90c-25e3-4add-ab77-08100cdee76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.919727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9a0c6d6e-7c38-4374-aeff-d568e68b3ecf None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.920956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9a0c6d6e-7c38-4374-aeff-d568e68b3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.930425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 778/1549] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.931623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d855-50d1-40c0-858a-3620f0bfa673 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.931911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d855-50d1-40c0-858a-3620f0bfa673 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.932189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d855-50d1-40c0-858a-3620f0bfa673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.938706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8872e90c-25e3-4add-ab77-08100cdee76b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.940003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8872e90c-25e3-4add-ab77-08100cdee76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.949311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 772/1550] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.950569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91c89-c6f6-4448-9c71-4a9f87fa85db None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.950882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91c89-c6f6-4448-9c71-4a9f87fa85db None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.951155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91c89-c6f6-4448-9c71-4a9f87fa85db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.952834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5823d855-50d1-40c0-858a-3620f0bfa673 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.954095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5823d855-50d1-40c0-858a-3620f0bfa673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.964539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 779/1551] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.969521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b307e-c364-45a8-af9b-e2d678018816 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.969814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b307e-c364-45a8-af9b-e2d678018816 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.970097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b307e-c364-45a8-af9b-e2d678018816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.972040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-62c91c89-c6f6-4448-9c71-4a9f87fa85db None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.973255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-62c91c89-c6f6-4448-9c71-4a9f87fa85db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:19.983229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 773/1552] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:19.984499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:19.984786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:19.985059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:19.993686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d19b307e-c364-45a8-af9b-e2d678018816 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:19.994942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d19b307e-c364-45a8-af9b-e2d678018816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.003130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 780/1553] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.004399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.004696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.004949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.009924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:20.011456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.020191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 774/1554] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.025257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.028181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:20.029410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.031401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07addd90-8a78-49b2-97a2-4f1dbe2dcba9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.036282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.038020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.038020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.038227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 781/1555] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.038375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.039328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.039619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.041592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58568c6d-37db-4057-8f4c-76b3d687b6af admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.052295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a604739-8696-4be2-ac6b-cb6f2d6c577e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.055486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 775/1556] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:20.055486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141eb0-581c-4971-873b-8a7470c7f701 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.055799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141eb0-581c-4971-873b-8a7470c7f701 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.055939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141eb0-581c-4971-873b-8a7470c7f701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.060393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.061146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.061310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.062173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.062486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.062674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.064195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.084335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5d141eb0-581c-4971-873b-8a7470c7f701 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:20.084637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 782/1557] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.084637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.084894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5d141eb0-581c-4971-873b-8a7470c7f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.087967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 776/1558] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.088890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535357b-22d3-432e-b3ee-336683d90e00 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.089296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.089597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.089872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.094191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.094924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.095275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.095620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.096351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.096724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.098419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445206f1-6488-4ca3-a073-4402477bbfa3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.111715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 783/1559] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.112990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.113314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.113563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.115347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:20.117330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.125909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 777/1560] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.126745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.135919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:20.136576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0707-7e60-494e-8aca-d6915748d88d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.137124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:20.141429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.142102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.142543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.142914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.143632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.143935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.144943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:20.144943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:16:20.144943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:16:20.145335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 784/1561] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:20.145831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.146152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.156867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f820f-95d5-4152-b14c-3ead6094fdea None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.160650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 778/1562] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.161648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.161767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.162266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.162603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.163412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.164073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.164379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.166262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.178971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 785/1563] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.179963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.185617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0881552-1abf-4e97-bce0-04d066ce8dd3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.189738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3029ed-7f95-4321-a62f-0da1c1608858 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.192843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.193452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.193756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.194212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.194954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.195212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.195573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.195857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.196153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.196767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.196887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.197795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:20.197795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:16:20.197795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:16:20.199237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.208203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 779/1564] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:20.209149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.221431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625291-8227-479f-a005-fa11bf235a9c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.227569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.228253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.228796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.228906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.229518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.229825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.231485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.243720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 780/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.244759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.254772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2b274-b0bc-40ce-b388-22037240a0b7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.259740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.260328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.260612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.260887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.261513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.263568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.362704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 786/1566] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:20.363615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.393180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:20.426186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10f345-9c81-42db-a222-50679c0c3776 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.433232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.433855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.434141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.434404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.435104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.435417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.437034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.440576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 781/1567] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/projects => generated 391 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:20.440576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.457456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 787/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.457456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.457775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:20.463417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f949c-f3d6-4a62-afd1-01b20f8f6479 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.507933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf353c2-1ab8-499c-a945-d727a7c129e5 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.512637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.513243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.513520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.513791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.514442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:20.514781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.516383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.520811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.521430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.521719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.522001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.522784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.524826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.528514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 782/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:20.529332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.539837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaa757-8fbc-4bbe-a2dd-2ffd6001b9e2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.545129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.559683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.559683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.559683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.559683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.559683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.649826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 788/1570] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:20.649826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.677531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 783/1571] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:20.679000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.713575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e22c5-0f48-4a82-b0f0-8f32fbb2b85d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.743580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1c58b-440a-4edc-af8f-2e9b5a0dd7ce admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:20.748410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.749018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.749295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.749570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.750189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.752117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.764292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:20.774149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:20.774870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:20.775156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:20.775434 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] PATH_INFO: `/v3/projects/aceecfa04cf941348d542c3e55569384/users/1d22ea3fb8fa4bd99537fb83279b2d24/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:20.791122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aceecfa04cf941348d542c3e55569384, user_id=1d22ea3fb8fa4bd99537fb83279b2d24, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:20.791896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:20.791896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:16:20.791896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:16:20.793668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:20.933848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 784/1572] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:20.934875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:20.972158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 789/1573] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:20 2022] PUT /identity/v3/projects/aceecfa04cf941348d542c3e55569384/users/1d22ea3fb8fa4bd99537fb83279b2d24/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:20.973173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.004913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e590e-b24b-454d-9fb3-4cb955dafce0 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.016469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:21.049101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6b868-b15d-49cc-99d6-78a2bfc2850e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.054298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.054974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.055251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.055528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.056172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.058130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.068617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.069907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.070186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.070454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.071113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.073089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.206481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 790/1574] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.207354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.217906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89014061-5bc9-4b4d-8274-7af98f88a8ec admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.222565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.223195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.223485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.223762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.224389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.227266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.252257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 785/1575] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:20 2022] POST /identity/v3/users => generated 302 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.258892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.263448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec995814-7032-4f35-b819-14417fc44e4a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.268185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.268771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.269046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.269322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.270074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.272398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.345253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 791/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.346329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.407442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20dfae-4442-4bbf-8e08-0d76f9587b0d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.412524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.413192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.413474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.413750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.414377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.417811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.425065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 786/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.433005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.487059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d6d224-6d55-4d48-a4ac-da1d3e27aff2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.492139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.492747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.507516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.507516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.507516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.507516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde95e73-395d-4051-bc06-327c3605ee80 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.615511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 792/1578] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.615511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.622901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e219-834f-4162-84f0-29a9f5471141 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.627506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.628094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.628377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.628657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.629277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.631744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.659076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 787/1579] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.664517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.664517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.664517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.680021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:21.681238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:21.688858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 788/1580] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:21.689611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.700170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84f23d-69d0-4629-bf8c-71599efbe64a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.708223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.708223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.708223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.708223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.708604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.708604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.718398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 789/1581] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:21.719214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.729122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62046444-0603-4121-ab8d-3088bb1ba5b9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.733796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.734376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.734682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.734959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.735612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.737237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.747755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 790/1582] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:21.765410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.765410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28713b61-374f-4671-a2e3-1fafcd2e2a8a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.765410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.765410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.765850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.765850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.765850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.766870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:21.829722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 793/1583] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:21.841569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:21.841569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb34a7-30e2-4093-9391-4dd0c9298284 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:21.846043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:21.846664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:21.846957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:21.847229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:21.847845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:21.849760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.047119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:22.053955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 794/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:22.058089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.064310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0006f0-c79f-4daa-8b5d-fb173451fbb2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.068864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.069471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.069709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.069988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.070646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.072244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.081909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 795/1585] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.082721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.092547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d65b4-a4d7-4ffe-945a-ee492caeff4a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.098171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.105906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 791/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:21 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:22.106047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.106047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.106047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.106354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.106445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.106445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.112813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 796/1587] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.113645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.124080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5061ffa-7208-4351-8271-abafeb0b7460 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.129352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.129946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.130227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.130499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.131160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.133036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.133552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:22.144311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 797/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.145138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.156573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e8077-7d50-4a1f-af15-5f300d2b6009 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.161247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.161841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.162118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.162219 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e200f0-ab14-45af-84dc-ec19dbfe7356 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.162385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.163552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.165148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.167275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.167888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.168185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.168446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.175174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 798/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.175944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.182953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.184407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.186858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3c77c-040e-4fbe-8539-6a997be69929 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.192179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.192783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.193064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.193349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.194244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 792/1590] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:22.194685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.195076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.196597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.204622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7934f-9a3c-47f6-83d3-abe6f8e3c209 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.207367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 799/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.208206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.213949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.214583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.214883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.215159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.215812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.217456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.220794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0c3d9-8e5e-4a6b-aadf-1f785c3fb01c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.227526 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.228154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.228435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.228821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.229488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.230545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 793/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.231358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.232897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.244966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c299320-a5ea-46af-964b-7415eca81839 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.246669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 800/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.246792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.252169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.254081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.254323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.254663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.255581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.257573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.258039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004bc2a3-e5cc-43de-b058-1d5c10be5d1b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.265524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.266247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.266466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.268307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.269026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.269772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 794/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.270510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.271670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.281510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ed7e6-1f7c-44eb-8b71-244193d694fe admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.289429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.290209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.291015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.291984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] PATH_INFO: `/v3/projects/84a2eee4447b4eadb79fbdbb874d4476/users/8d154e96376a49918658b0c340c722e9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.292120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 801/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.293105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.308379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac38c68-c7a4-457a-9d92-93360e6d182a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.313745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a2eee4447b4eadb79fbdbb874d4476, user_id=8d154e96376a49918658b0c340c722e9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.313929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.314438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.314759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.315038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] PATH_INFO: `/v3/projects/80aa4a693a65446eb69a5957995f11b4/users/f81ff055d2ca4618b4b29fba5b8cfad1/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.317003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.331598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] RBAC: Authorizing `identity:create_grant(project_id=80aa4a693a65446eb69a5957995f11b4, user_id=f81ff055d2ca4618b4b29fba5b8cfad1, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.334517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.423755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 795/1596] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/84a2eee4447b4eadb79fbdbb874d4476/users/8d154e96376a49918658b0c340c722e9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:22.424751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.479191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 802/1597] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/80aa4a693a65446eb69a5957995f11b4/users/f81ff055d2ca4618b4b29fba5b8cfad1/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:22.480221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.493613 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9ff86-9cc0-40f1-94e4-2611b7781702 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.554117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a680f50-8ad3-444b-9aa9-a1693144afec admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.559601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.560210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.560573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.560803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.561517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.563197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.570288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.570288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.570288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.570288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.570754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.570754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.573932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 803/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.577907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 796/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.578122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.578804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.589582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bc9df-45af-48fc-b111-a9ae9a3ddf7d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.592461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774b6be-085b-480c-ba09-029764d79b8f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.596938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.597210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.597622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.597821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.597936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.598110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.598273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] PATH_INFO: `/v3/projects/2ce4bdd3a2a3450c9f93565510c79db5/users/04958f2f938d4ca5b8f91c2ba69bfa5f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.598378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.599141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.600834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.612090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 797/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:22.613001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.623722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce4bdd3a2a3450c9f93565510c79db5, user_id=04958f2f938d4ca5b8f91c2ba69bfa5f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.625999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06146c3-c913-47db-840c-2072af842302 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.626473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c0d25-fcac-4761-90cb-3ae393f57c7d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.631024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.632323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.632602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.632880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] PATH_INFO: `/v3/projects/bf6936f23aa446008e8119807ec8724d/users/a7b7684d766542b097b88d642114023b/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.644582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:22.648814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6936f23aa446008e8119807ec8724d, user_id=a7b7684d766542b097b88d642114023b, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.651560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.732975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 804/1601] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/2ce4bdd3a2a3450c9f93565510c79db5/users/04958f2f938d4ca5b8f91c2ba69bfa5f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:22.735357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7dac-757f-4a95-83e5-664d53f67764 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.735357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7dac-757f-4a95-83e5-664d53f67764 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.735357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c7dac-757f-4a95-83e5-664d53f67764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.762313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-737c7dac-757f-4a95-83e5-664d53f67764 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:22.763558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-737c7dac-757f-4a95-83e5-664d53f67764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:22.788625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:22.826295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 805/1602] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:22.827795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.828258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.828554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.837933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 798/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/bf6936f23aa446008e8119807ec8724d/users/a7b7684d766542b097b88d642114023b/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:22.838376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.860781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] MFA Rules not processed for user `f81ff055d2ca4618b4b29fba5b8cfad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:22.862047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:22.912287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b426fa2-ecd2-4fcd-9ad9-50eb3c9aa0ae admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.916155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.916823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.917172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.917512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] PATH_INFO: `/v3/projects/b1b11d5ed90b49e29785f7df4b2ddc48/users/8edb7f9dc184486e8fd296fb37898311/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.934512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b11d5ed90b49e29785f7df4b2ddc48, user_id=8edb7f9dc184486e8fd296fb37898311, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.937508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:22.945656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 806/1604] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:22.946692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:22.958232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3cf05-493b-4564-86ef-c2ff02f23f43 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:22.964502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:22.964502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:22.964502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:22.964502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] PATH_INFO: `/v3/projects/bdfe27a09bb6426a84d193f9dda3c43f/users/b7aeee13c8a84f0aa29f9190680745c9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:22.981379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfe27a09bb6426a84d193f9dda3c43f, user_id=b7aeee13c8a84f0aa29f9190680745c9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:22.984120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3495d-8b77-438d-803a-70c1b1bfffd0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.094017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 799/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/b1b11d5ed90b49e29785f7df4b2ddc48/users/8edb7f9dc184486e8fd296fb37898311/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:23.094920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.133828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 807/1606] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:22 2022] PUT /identity/v3/projects/bdfe27a09bb6426a84d193f9dda3c43f/users/b7aeee13c8a84f0aa29f9190680745c9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:23.139751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.139751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.139751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.168150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] MFA Rules not processed for user `04958f2f938d4ca5b8f91c2ba69bfa5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:23.169422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:23.180405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eea143-f1be-4b91-be45-029d805ff7bb admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:23.245506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:23.246179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.246770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.246896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] PATH_INFO: `/v3/projects/c55c4050ef494e8f914271798c0e901b/users/6c1aacb78343435aa348e148a7605b18/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.250860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 808/1607] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:23.251689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.267993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55c4050ef494e8f914271798c0e901b, user_id=6c1aacb78343435aa348e148a7605b18, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:23.270844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94740d-7bf6-4efe-b3f2-43ca85698fee admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.287429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:23.329848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90a2c-cf04-4dce-ab1e-f3057e50c30c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:23.334057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:23.335330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.335830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.336117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.345572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:23.347009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab27c78-06ce-468a-bb15-319a0892a8ba service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.357449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 809/1608] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:23.358883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a100-f4cb-4d08-9280-733343047308 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.359174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a100-f4cb-4d08-9280-733343047308 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.359449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a100-f4cb-4d08-9280-733343047308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.378863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-4801a100-f4cb-4d08-9280-733343047308 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:23.380075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4801a100-f4cb-4d08-9280-733343047308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:23.388094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 810/1609] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:23.389839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801a100-f4cb-4d08-9280-733343047308 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.399513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801a100-f4cb-4d08-9280-733343047308 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:23.404239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:23.404889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.405569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.405856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.406490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:23.408433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.413185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 800/1610] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:23 2022] PUT /identity/v3/projects/c55c4050ef494e8f914271798c0e901b/users/6c1aacb78343435aa348e148a7605b18/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:23.414689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e2a9-1bad-451a-a1a8-aae3f193d7d3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.424385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e2a9-1bad-451a-a1a8-aae3f193d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.424385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45e2a9-1bad-451a-a1a8-aae3f193d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.424385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:23.448829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3c45e2a9-1bad-451a-a1a8-aae3f193d7d3 None None] MFA Rules not processed for user `8edb7f9dc184486e8fd296fb37898311`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:23.450083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3c45e2a9-1bad-451a-a1a8-aae3f193d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:23.523872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 801/1611] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:23.525719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.526014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.526291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.558055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] MFA Rules not processed for user `b7aeee13c8a84f0aa29f9190680745c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:23.559433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:23.576036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 811/1612] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/projects => generated 405 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:23.577034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.645835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 802/1613] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:23.645835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.659502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15c117-a789-40e2-beaa-7d15b7b76f02 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:23.666751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:23.667440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.667734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.668613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.669402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:23.671439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.711392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81e736-308c-4401-9610-bbc3653a2286 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:23.765050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:23.765667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.765953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.766327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.775550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:23.777136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39fd84-1ef1-4d33-bf4f-278ab346034f service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:23.786865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 803/1614] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:23.789312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:23.789637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:23.789953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:23.824289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] MFA Rules not processed for user `6c1aacb78343435aa348e148a7605b18`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:23.825517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:23.880362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 812/1615] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/projects => generated 387 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:23.884765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.905288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 804/1616] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:23.907005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:23.930481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173e428-779e-4b9b-8acd-028bf59ff835 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.014544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:24.018939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe287e5-e6c4-46bc-a9b2-d4026e74db00 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.025132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.025762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.026052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.026333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.027217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.030537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4560bbb-c0dc-4d4f-8287-db5805168afa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.041043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.041765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.042118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.042462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.043968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.046604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339c99-8237-4e1e-b8c6-cf7c72ffcb36 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.190041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 813/1617] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/projects => generated 371 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:24.194896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7c0d-ccac-4de8-b7c6-4943a18b84a2 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.195542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7c0d-ccac-4de8-b7c6-4943a18b84a2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.195843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7c0d-ccac-4de8-b7c6-4943a18b84a2 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.197502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 814/1618] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 06:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:24.198551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a7c0d-ccac-4de8-b7c6-4943a18b84a2 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.212622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:24.279919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a7c0d-ccac-4de8-b7c6-4943a18b84a2 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.284456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.285727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.286031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.286300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.287112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.290018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d625c6-bc1d-47cf-a279-daf375fd1c46 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.306804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 805/1619] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:23 2022] POST /identity/v3/users => generated 309 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:24.308780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5eb9-3d09-4283-abf3-613d1d357613 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.309187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5eb9-3d09-4283-abf3-613d1d357613 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.309563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5eb9-3d09-4283-abf3-613d1d357613 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.311322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 806/1620] 10.222.0.40 () {54 vars in 962 bytes} [Fri Aug 26 06:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:24.312145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5eb9-3d09-4283-abf3-613d1d357613 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.338960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5eb9-3d09-4283-abf3-613d1d357613 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.344249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.344249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.344249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.344249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.355650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.358212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2f96d-8d4f-4953-9933-1de5ae263654 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.425202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 815/1621] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:24 2022] POST /identity/v3/users => generated 300 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:24.444412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.444412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.444412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.482013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:24.483542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:24.589806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 807/1622] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:24 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:24.590856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726710-9eca-431f-b217-40bd6553856c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.591147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726710-9eca-431f-b217-40bd6553856c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.591612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726710-9eca-431f-b217-40bd6553856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.596947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 816/1623] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 06:16:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:24.597849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.629892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad8e5f-3f14-4a5f-bec4-70b08e023659 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.640818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.641511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.641797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.642075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.642963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.645856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-23726710-9eca-431f-b217-40bd6553856c None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:24.647775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.649445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-23726710-9eca-431f-b217-40bd6553856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:24.669830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 817/1624] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:24.671560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.673576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 808/1625] 10.222.0.40 () {58 vars in 1059 bytes} [Fri Aug 26 06:16:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:24.674778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23726710-9eca-431f-b217-40bd6553856c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.696794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba318e-c5fe-466a-b9f9-d03b96913d8a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.699352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23726710-9eca-431f-b217-40bd6553856c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.705488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.705704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.705704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.705980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.706299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.707001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.708838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.711402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.712046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.712653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.713791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.720730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.723791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 809/1626] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:24.724955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.742541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 818/1627] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:24.743929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.744880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68ea08-9531-4ce0-85bc-a8481cd15445 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.754288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.761724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.761724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.761724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.762226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d574c2-a5ba-4e42-8147-815af069522a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.766240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.767329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.767651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.768149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] PATH_INFO: `/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7/users/ad0f32ff72ec482ca07ac3fcd1ec758f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.817866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132e00c47198494c9cb7fbe1b0a45fb7, user_id=ad0f32ff72ec482ca07ac3fcd1ec758f, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.826167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.886695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:24.888646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:24.913794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 810/1628] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:24.914875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:24.940135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371bace-0c02-445e-aa81-cad3703fc1fe service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:24.956499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:24.957204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:24.957554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:24.957799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:24.979971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 819/1629] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:24 2022] PUT /identity/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7/users/ad0f32ff72ec482ca07ac3fcd1ec758f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:24.980801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:25.088954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.093183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.114584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 811/1630] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:25.116751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:25.121491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be40b9-42e2-4da1-95fd-ee77f24d9795 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:25.134200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:25.138524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.138524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.138524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] PATH_INFO: `/v3/projects/2d52f9f292e24326aa80e3490cdbb370/users/d02467ec4b9b46ae91d4467073a79768/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.138844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0c3e-9ded-41cd-86ff-2c621a74374f service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:25.150005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:25.152713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.153033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.153378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] PATH_INFO: `/v3/projects/f7991305a156454d9137d71445bf52b6/users/a8f4c9dbdc2c46df896e51a340cb1dcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.170576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d52f9f292e24326aa80e3490cdbb370, user_id=d02467ec4b9b46ae91d4467073a79768, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.173642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.182822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7991305a156454d9137d71445bf52b6, user_id=a8f4c9dbdc2c46df896e51a340cb1dcf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.185787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d7fb3-c7d4-4ac2-a7c6-9e00d3db1684 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.302466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 820/1631] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:24 2022] PUT /identity/v3/projects/2d52f9f292e24326aa80e3490cdbb370/users/d02467ec4b9b46ae91d4467073a79768/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:25.304494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:25.407072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 812/1632] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:25 2022] PUT /identity/v3/projects/f7991305a156454d9137d71445bf52b6/users/a8f4c9dbdc2c46df896e51a340cb1dcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:25.407072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3cb7-eace-4cc3-8407-5f7218847d57 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.407072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3cb7-eace-4cc3-8407-5f7218847d57 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.407545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d3cb7-eace-4cc3-8407-5f7218847d57 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.409425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 813/1633] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 06:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:25.409959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d3cb7-eace-4cc3-8407-5f7218847d57 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:25.451839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfabadb9-bb04-434c-86f7-e58f5299228f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:25.481263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:25.547704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d3cb7-eace-4cc3-8407-5f7218847d57 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:25.554216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:25.556953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.556953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.556953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.556953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.557838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.572570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:25.573403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.573691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.573983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.574642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.576358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af98cc-4d4e-43b2-b600-6abe79322d07 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.577356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 814/1634] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:25.578030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:25.589855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 821/1635] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:25.591361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.591663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.591838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.593276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66344911-efb6-4968-998f-89353b16d84e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:25.599394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:25.600216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.600502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:25.600925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:25.601716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:25.603581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2dcb7-4436-43a9-95b0-599316dfe850 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:25.612887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:25.614243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:25.624410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 815/1636] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:25.625711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7025d-3725-40f0-8c69-8b0630df24a4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:25.626001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7025d-3725-40f0-8c69-8b0630df24a4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.251313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7025d-3725-40f0-8c69-8b0630df24a4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.251313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 816/1637] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 06:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:26.251313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7025d-3725-40f0-8c69-8b0630df24a4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:26.251313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7025d-3725-40f0-8c69-8b0630df24a4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:26.254680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:26.254680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.254680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.254680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] PATH_INFO: `/v3/projects/2d52f9f292e24326aa80e3490cdbb370/users/d02467ec4b9b46ae91d4467073a79768/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.258704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d52f9f292e24326aa80e3490cdbb370, user_id=d02467ec4b9b46ae91d4467073a79768, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:26.258704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:26.258704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 817/1638] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:25 2022] PUT /identity/v3/projects/2d52f9f292e24326aa80e3490cdbb370/users/d02467ec4b9b46ae91d4467073a79768/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:26.258704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:26.260828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 822/1639] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:26.260828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:26.260828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6054a2-34f6-442e-9935-b1138616116a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:26.261659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2cd39-be89-4141-aa45-be8d16a9c326 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:26.261659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:26.261659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.261659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.262922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] PATH_INFO: `/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7/users/ad0f32ff72ec482ca07ac3fcd1ec758f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.262922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=132e00c47198494c9cb7fbe1b0a45fb7, user_id=ad0f32ff72ec482ca07ac3fcd1ec758f, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:26.262922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf9cc65-a36b-4df3-8add-2c5dcb81eddb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:26.263866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:26.263866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.263866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.263866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] PATH_INFO: `/v3/projects/f7991305a156454d9137d71445bf52b6/users/a8f4c9dbdc2c46df896e51a340cb1dcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.264587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7991305a156454d9137d71445bf52b6, user_id=a8f4c9dbdc2c46df896e51a340cb1dcf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:26.264587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18205a8-4d3d-4a3e-bdac-7b901ff5535e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:26.265171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 818/1640] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:25 2022] PUT /identity/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7/users/ad0f32ff72ec482ca07ac3fcd1ec758f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:26.265171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.265171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.265171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.266260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 823/1641] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:25 2022] PUT /identity/v3/projects/f7991305a156454d9137d71445bf52b6/users/a8f4c9dbdc2c46df896e51a340cb1dcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:26.266444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:26.266605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490351-5ce2-4cab-9a10-990a320e5f79 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.266605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490351-5ce2-4cab-9a10-990a320e5f79 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.267042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:26.267226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490351-5ce2-4cab-9a10-990a320e5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.267226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-aa490351-5ce2-4cab-9a10-990a320e5f79 None None] MFA Rules not processed for user `d02467ec4b9b46ae91d4467073a79768`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:26.267226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-aa490351-5ce2-4cab-9a10-990a320e5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:26.267677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 819/1642] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:16:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:26.267677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 824/1643] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] MFA Rules not processed for user `ad0f32ff72ec482ca07ac3fcd1ec758f`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:26.268074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:26.268917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb190c-d921-4a0b-a4c2-6b09f4cfa189 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:26.268917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:26.268917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.268917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80027fb3-a489-4b9b-947a-f77063b45d8f service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 820/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.269475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.270314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.270314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] MFA Rules not processed for user `a8f4c9dbdc2c46df896e51a340cb1dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:26.276094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:26.305771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 825/1645] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:26 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:26.307643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:26.338749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931809f-6570-4638-bcf5-1e4136c4f660 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:26.350226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:26.351181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:26.351467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:26.351746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:26.385584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:26.387100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:26.416446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 826/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:26.441427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 821/1647] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:27.152090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:27.950395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:28.242787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:28.252821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ad327-4202-4aa8-bf11-5b12fdd86f47 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:28.263894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:28.264469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:28.264637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:28.265061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:28.282598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:28.284422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:28.299971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 827/1648] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:28.468055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:28.476253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:28.493432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5636b-f678-454d-a0b3-81498c61864d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:28.493740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa01e4-4792-454f-bf06-9975ffdf4d0a service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:28.504919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:28.506297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:28.507069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:28.512036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:28.512425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:28.513048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:28.513382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:28.513701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:28.542670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:28.545690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:28.549817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:28.556763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:28.581596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 822/1649] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:28.609054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 828/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:29.275657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:29.469807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6aa1e-bc8d-4621-84fb-47aa32e07044 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:29.490154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:29.490154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:29.492837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:29.495213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] PATH_INFO: `/v3/users/04958f2f938d4ca5b8f91c2ba69bfa5f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:29.498112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04958f2f938d4ca5b8f91c2ba69bfa5f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:29.499517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:29.513912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b15f78-43cd-4b89-a367-149ce8989006 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:29.516674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:29.533267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:29.534865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:29.536135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:29.550732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:29.561680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:29.768499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:29.771224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:29.787352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 829/1651] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:29.789190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:29.821504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4834a-9fb7-45d9-921c-5a2f6a6e9af7 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:29.827818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:29.828882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:29.830609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:29.831140 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] PATH_INFO: `/v3/users/f81ff055d2ca4618b4b29fba5b8cfad1` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:29.832689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81ff055d2ca4618b4b29fba5b8cfad1)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:29.835613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: WARNING py.warnings [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:29.835613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: warnings.warn(msg) Aug 26 06:16:29.835613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: Aug 26 06:16:29.837390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:29.918049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04958f2f938d4ca5b8f91c2ba69bfa5f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:29.920684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04958f2f938d4ca5b8f91c2ba69bfa5f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:29.965995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04958f2f938d4ca5b8f91c2ba69bfa5f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.152158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04958f2f938d4ca5b8f91c2ba69bfa5f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.176551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6bbd6241-b837-4774-ac86-ed654079b2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04958f2f938d4ca5b8f91c2ba69bfa5f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.203009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 823/1652] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:29 2022] DELETE /identity/v3/users/04958f2f938d4ca5b8f91c2ba69bfa5f => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:30.207032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:30.207329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:30.207607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:30.264746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:30.266223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:30.443092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81ff055d2ca4618b4b29fba5b8cfad1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.444822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81ff055d2ca4618b4b29fba5b8cfad1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.452789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 824/1653] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:30.465198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:30.511588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81ff055d2ca4618b4b29fba5b8cfad1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.557379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:30.559062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:30.688162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386134a-bb36-4e0e-9a8a-bf8ec15683e4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:30.699564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:30.700259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:30.700554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:30.700811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:30.704359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:30.709978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81ff055d2ca4618b4b29fba5b8cfad1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.774968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-858f1c08-a40f-42a3-9e2e-28a4739d129b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81ff055d2ca4618b4b29fba5b8cfad1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:30.791641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:30.793820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:30.795142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 830/1654] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:29 2022] DELETE /identity/v3/users/f81ff055d2ca4618b4b29fba5b8cfad1 => generated 0 bytes in 1003 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:30.796706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:30.797152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:30.797575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:30.808398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 825/1655] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:30.823026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:30.824547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:30.951896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 831/1656] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:31.120361 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:31.122940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:31.160043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1808a6-50f6-4db1-9d20-ce31815e5df7 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:31.382970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:31.384461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:31.384941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:31.385415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:31.427801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:31.437372 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:31.494650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 826/1657] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:32.642753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:32.645022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:32.648327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:32.679934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b02fa4-0a83-445b-a168-064d8880f991 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:32.705469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:32.706148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:32.706497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:32.706754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] PATH_INFO: `/v3/users/6c1aacb78343435aa348e148a7605b18` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:32.709160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1aacb78343435aa348e148a7605b18)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:32.711456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:33.066389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1aacb78343435aa348e148a7605b18'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.072819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1aacb78343435aa348e148a7605b18'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.099020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1aacb78343435aa348e148a7605b18'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.138603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:33.140252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:33.170718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540a88-b60b-4017-9da9-d75cb5375a0c service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:33.181858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:33.182774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:33.183063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:33.183340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] PATH_INFO: `/v3/projects/2ce4bdd3a2a3450c9f93565510c79db5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:33.189102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce4bdd3a2a3450c9f93565510c79db5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:33.190221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1aacb78343435aa348e148a7605b18'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.190561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: WARNING py.warnings [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 06:16:33.190561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: warnings.warn(msg) Aug 26 06:16:33.190561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: Aug 26 06:16:33.191894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:33.200655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-479991c3-f286-4ca2-a6a2-c7f17d375bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1aacb78343435aa348e148a7605b18'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.227313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 832/1658] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:32 2022] DELETE /identity/v3/users/6c1aacb78343435aa348e148a7605b18 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:33.230684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:33.231817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:33.232608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:33.279426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:33.281507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:33.418096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce4bdd3a2a3450c9f93565510c79db5', 'request_id': 'req-54910e8e-5634-460a-8af1-81be5fd66b5b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.427874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 833/1659] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:33.457698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:33.459029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:33.504860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:33.602761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69def7-dac4-4fac-b351-5b0363a50289 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:33.640338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce4bdd3a2a3450c9f93565510c79db5', 'request_id': 'req-54910e8e-5634-460a-8af1-81be5fd66b5b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.651686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce4bdd3a2a3450c9f93565510c79db5', 'request_id': 'req-54910e8e-5634-460a-8af1-81be5fd66b5b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.707436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] The token cache is being invalidate because project 2ce4bdd3a2a3450c9f93565510c79db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:33.709174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.715585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:33.724677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 827/1660] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:33 2022] DELETE /identity/v3/projects/2ce4bdd3a2a3450c9f93565510c79db5 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:33.731688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:33.734730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:33.757778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:33.762532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:33.762532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:33.762532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:33.762969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:33.938320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54910e8e-5634-460a-8af1-81be5fd66b5b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:33.959406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:33.960382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:33.960666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:33.960929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] PATH_INFO: `/v3/users/a8f4c9dbdc2c46df896e51a340cb1dcf` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:33.962782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f4c9dbdc2c46df896e51a340cb1dcf)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:33.964994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:34.043767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:34.045263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:34.094304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 834/1661] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:16:34.338701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f4c9dbdc2c46df896e51a340cb1dcf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:34.339572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f4c9dbdc2c46df896e51a340cb1dcf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:34.368066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f4c9dbdc2c46df896e51a340cb1dcf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:34.516500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f4c9dbdc2c46df896e51a340cb1dcf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:34.526529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f4c9dbdc2c46df896e51a340cb1dcf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:34.534155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 828/1662] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:33 2022] DELETE /identity/v3/users/a8f4c9dbdc2c46df896e51a340cb1dcf => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:34.750822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:34.755291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:34.772293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3d92-122a-4fd5-aed2-2cc7a71d72d7 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:34.918004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:34.919427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:34.920072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:34.921271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] PATH_INFO: `/v3/projects/80aa4a693a65446eb69a5957995f11b4` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:34.924527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80aa4a693a65446eb69a5957995f11b4)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:34.929942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:35.020009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:35.171534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80aa4a693a65446eb69a5957995f11b4', 'request_id': 'req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.188286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80aa4a693a65446eb69a5957995f11b4', 'request_id': 'req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.362072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80aa4a693a65446eb69a5957995f11b4', 'request_id': 'req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.388149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] The token cache is being invalidate because project 80aa4a693a65446eb69a5957995f11b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:35.388442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.391252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.393083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 835/1663] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:34 2022] DELETE /identity/v3/projects/80aa4a693a65446eb69a5957995f11b4 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:35.398172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:35.404533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:35.438406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bbe43-7d1f-447d-b67d-c1a4b5ef66e3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:35.577120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:35.578043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:35.578468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:35.579047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] PATH_INFO: `/v3/users/ad0f32ff72ec482ca07ac3fcd1ec758f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:35.580488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0f32ff72ec482ca07ac3fcd1ec758f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:35.583392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:35.851444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0f32ff72ec482ca07ac3fcd1ec758f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.852383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0f32ff72ec482ca07ac3fcd1ec758f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:35.908356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0f32ff72ec482ca07ac3fcd1ec758f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.012196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:36.014060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:36.025165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0f32ff72ec482ca07ac3fcd1ec758f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.039819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9dac1849-571a-4776-9252-0e5a9764aa75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0f32ff72ec482ca07ac3fcd1ec758f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.061983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa2ae9-9e7b-4e9a-b8a7-84d2069621f6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:36.084378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 829/1664] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:35 2022] DELETE /identity/v3/users/ad0f32ff72ec482ca07ac3fcd1ec758f => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:36.102886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:36.103642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:36.104000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:36.104347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] PATH_INFO: `/v3/projects/f7991305a156454d9137d71445bf52b6` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:36.106152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7991305a156454d9137d71445bf52b6)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:36.108950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:36.255559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:36.255559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:36.255559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:36.319819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] MFA Rules not processed for user `1d22ea3fb8fa4bd99537fb83279b2d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:36.321485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:36.406012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7991305a156454d9137d71445bf52b6', 'request_id': 'req-60cff1f2-1b97-45ae-8dda-26e1d61db804'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.425831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7991305a156454d9137d71445bf52b6', 'request_id': 'req-60cff1f2-1b97-45ae-8dda-26e1d61db804'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.495190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 830/1665] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:36.495817 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:36.498304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:36.522306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7991305a156454d9137d71445bf52b6', 'request_id': 'req-60cff1f2-1b97-45ae-8dda-26e1d61db804'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.556115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] The token cache is being invalidate because project f7991305a156454d9137d71445bf52b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:36.557878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.561899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:36.571790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 836/1666] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:36 2022] DELETE /identity/v3/projects/f7991305a156454d9137d71445bf52b6 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:36.573202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:36.574941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:36.709848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45c0c0-050e-43d9-bffd-371435c6a4d7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:36.766051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cff1f2-1b97-45ae-8dda-26e1d61db804 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:36.793416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:36.794307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:36.794760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:36.795714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] PATH_INFO: `/v3/projects/c55c4050ef494e8f914271798c0e901b` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:36.798312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55c4050ef494e8f914271798c0e901b)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:36.800486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:36.865579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:36.866707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:36.867157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:36.867566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:36.875634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:36.988678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:37.025780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55c4050ef494e8f914271798c0e901b', 'request_id': 'req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.038156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:37.039269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55c4050ef494e8f914271798c0e901b', 'request_id': 'req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.042305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b49a-11f2-4f0f-a5fb-1eb690c91124 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:37.057820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 831/1667] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:16:37.059565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5372ae-d2ad-4593-8b5b-6eecec3c87bb None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.060053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5372ae-d2ad-4593-8b5b-6eecec3c87bb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.061227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5372ae-d2ad-4593-8b5b-6eecec3c87bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.097835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7f5372ae-d2ad-4593-8b5b-6eecec3c87bb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:37.099350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7f5372ae-d2ad-4593-8b5b-6eecec3c87bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.169239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55c4050ef494e8f914271798c0e901b', 'request_id': 'req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.229516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] The token cache is being invalidate because project c55c4050ef494e8f914271798c0e901b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:37.232272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.235829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4ca835fe-d378-4c50-8f46-a0d786ddb7ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.241092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 837/1668] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:36 2022] DELETE /identity/v3/projects/c55c4050ef494e8f914271798c0e901b => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:37.245922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a33fb6-4059-4008-9fe7-4373da02cdf6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.246447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a33fb6-4059-4008-9fe7-4373da02cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.246890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a33fb6-4059-4008-9fe7-4373da02cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.296092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-24a33fb6-4059-4008-9fe7-4373da02cdf6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:37.297541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-24a33fb6-4059-4008-9fe7-4373da02cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.316674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 832/1669] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:37.318203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.318495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.319619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.347305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 838/1670] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:37.356301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.356917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.357406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.418698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:37.419923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.428001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:37.429370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.461652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 833/1671] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:37.472578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:37.478395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.488221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 839/1672] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:37.491577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:37.518426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e7423-0175-4cda-8b3c-b400745865fe None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:37.523860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115dea66-8cc3-40c1-80c5-083f1164f22e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:37.535755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:37.535934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:37.536582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.536884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.537220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.538024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:37.538256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.538691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:37.538854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.539259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] PATH_INFO: `/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.541074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=132e00c47198494c9cb7fbe1b0a45fb7)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:37.543378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:37.543662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:37.579931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 840/1673] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:37.579931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:37.601127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc617-fd7f-42fa-874c-bc30beb7ebc2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:37.617210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:37.617848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.618142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.618427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.619116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:37.619709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:37.621022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be999683-066e-4f4c-b5b5-fd0cb36dc923 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:37.648297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 841/1674] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:37.649880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:37.650583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:37.651047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:37.706605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] MFA Rules not processed for user `8d154e96376a49918658b0c340c722e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:37.708175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:37.807217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132e00c47198494c9cb7fbe1b0a45fb7', 'request_id': 'req-ad4162dd-337e-4b09-adba-b72c0693d9a2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:37.868039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 842/1675] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:37.869558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:37.901707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5008794-d3f4-491e-930c-bc7ac3c82ab8 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.030244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132e00c47198494c9cb7fbe1b0a45fb7', 'request_id': 'req-ad4162dd-337e-4b09-adba-b72c0693d9a2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:38.039923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132e00c47198494c9cb7fbe1b0a45fb7', 'request_id': 'req-ad4162dd-337e-4b09-adba-b72c0693d9a2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:38.052800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.053495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.053776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.054849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.055731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.057955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1eddb-a056-4dbe-9f50-9ef7ff44dd86 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.066473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] The token cache is being invalidate because project 132e00c47198494c9cb7fbe1b0a45fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:38.066803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:38.067514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:38.075978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 834/1676] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:37 2022] DELETE /identity/v3/projects/132e00c47198494c9cb7fbe1b0a45fb7 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:38.077260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.079412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.102574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4162dd-337e-4b09-adba-b72c0693d9a2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.109030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.109907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.111426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.111957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.112894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.115608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1d7a1-3955-4ad4-a911-5e8ca5811615 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.211275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 843/1677] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:37 2022] POST /identity/v3/projects => generated 379 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:38.211275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94c89-7f77-4998-8681-8a5e6c25352b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.211275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94c89-7f77-4998-8681-8a5e6c25352b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.211275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc94c89-7f77-4998-8681-8a5e6c25352b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.231898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-cfc94c89-7f77-4998-8681-8a5e6c25352b None None] MFA Rules not processed for user `8d154e96376a49918658b0c340c722e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:38.233152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cfc94c89-7f77-4998-8681-8a5e6c25352b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.297095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 835/1678] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/projects => generated 367 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:38.297689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.298066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.298399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.314024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 844/1679] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:38.316028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.317499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.318068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.332793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:38.334041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.353952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] MFA Rules not processed for user `a7b7684d766542b097b88d642114023b`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:38.355399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.465341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 845/1680] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:38.466477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.468249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.480614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:38.505028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 836/1681] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:38.506251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.527986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6049ad-d554-4f20-b7ea-a302ae423071 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.534882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.535499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.535930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.536682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.538227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.541899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.574720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dec68-4129-44b0-87e6-a51bd2f6f57f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.580729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.581492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.581804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.583513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.583513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.585978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a77da-0840-49c9-827d-fd66baadc4a9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.671229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 837/1682] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:38.672512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.679295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.725464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 846/1683] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:38.725464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d0a8-fa16-40a9-a198-fd4eb4432c47 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.725464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d0a8-fa16-40a9-a198-fd4eb4432c47 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.725464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4d0a8-fa16-40a9-a198-fd4eb4432c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.759254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-71e4d0a8-fa16-40a9-a198-fd4eb4432c47 None None] MFA Rules not processed for user `a7b7684d766542b097b88d642114023b`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:38.760446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-71e4d0a8-fa16-40a9-a198-fd4eb4432c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.775351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 847/1684] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:38.777132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.777564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.778146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.798326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc3eb-929b-4ab3-b752-25abf6fb518b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.813861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.815528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.815966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.816591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.819105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:38.819669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.820593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.841582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 848/1685] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:38.841582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.851820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f13c9-89bd-4c50-9ad8-b85e6d3ec9ea None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.857244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.857991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.858379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.858857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.859735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.861751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.878925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 849/1686] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:38.879863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.910262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de029ff-b4a9-4396-b61e-816e6cd7b45a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:38.925527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:38.927669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:38.929978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:38.930302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:38.931049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.932738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:38.958484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 850/1687] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:38.965033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:38.970042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:38.992726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:38.996619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.009413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10239428-417a-4338-8ff4-e039af137731 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.019877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 838/1688] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:39.024523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.025154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.025408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.026305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.030053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.047895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.052263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.054014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a7dd4-3428-4c91-ad34-f45f4cea9675 service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.060371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.061273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.061794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.062214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.063078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:39.063545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.065638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.076340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 851/1689] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:39.077314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.087504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 839/1690] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:39.091910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.101178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1aa20-076a-44f3-b00c-994933e9c85b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.106682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.107368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.107516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490386e5-c8cb-47be-ac6f-0a49de797fbe service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.107908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.108195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.109272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.119258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.124326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.128108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.128444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.128727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.130701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.132568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.143163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 840/1691] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:39.145409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.158783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 852/1692] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:39.159838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.188135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f69d-b895-4bf9-beeb-3291a5ef810c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.188135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.188135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.188135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.188487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] PATH_INFO: `/v3/projects/92beeeda046c4ce88fb959efab32ea8c/users/84d8b95368f2474e9193e166fcce977c/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.188636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cc333-4a63-4494-bba2-844b81712035 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.198236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.200715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.201192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.201679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.204736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=92beeeda046c4ce88fb959efab32ea8c, user_id=84d8b95368f2474e9193e166fcce977c, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.209914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.210832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.213025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854595b-470e-431d-b4bb-1a0af7a848fa admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.373029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 853/1693] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:39 2022] PUT /identity/v3/projects/92beeeda046c4ce88fb959efab32ea8c/users/84d8b95368f2474e9193e166fcce977c/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:39.374103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.430234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 841/1694] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:39 2022] POST /identity/v3/projects => generated 403 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:39.432076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.434494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.438212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.513623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] MFA Rules not processed for user `84d8b95368f2474e9193e166fcce977c`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:39.516735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:39.553793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832554a-ca90-49c8-86a9-6f2b33590aaf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.702420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.702921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.703210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.704159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] PATH_INFO: `/v3/projects/bb6fd7ac40454d578844c82354011594/users/b9a0adcfed5c4a8ba9366a1a69945e0c/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.722596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 842/1695] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:39.723707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.742265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb6fd7ac40454d578844c82354011594, user_id=b9a0adcfed5c4a8ba9366a1a69945e0c, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.744935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.758278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbf79a-df92-4a61-a707-770acca51ec2 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:39.775111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:39.776261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.776651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.777181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.777994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:39.780182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d7941-f4c5-48b1-bfcc-c7dd9781806d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:39.939744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 854/1696] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:39 2022] PUT /identity/v3/projects/bb6fd7ac40454d578844c82354011594/users/b9a0adcfed5c4a8ba9366a1a69945e0c/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:39.939744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:39.948225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 843/1697] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:39 2022] POST /identity/v3/users => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:39.948225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4a36-a6fe-417f-8db3-787038d6774f None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.948225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4a36-a6fe-417f-8db3-787038d6774f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.948225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4a36-a6fe-417f-8db3-787038d6774f None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.954925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 844/1698] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 06:16:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:39.956652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aefc9-d5e3-4b12-a89c-d9530c28c366 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.957460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aefc9-d5e3-4b12-a89c-d9530c28c366 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.957748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aefc9-d5e3-4b12-a89c-d9530c28c366 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:39.960190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 845/1699] 10.222.0.40 () {54 vars in 912 bytes} [Fri Aug 26 06:16:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:39.962436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768abb2d-cf1d-4921-8eb7-3c443e65c251 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:39.963235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768abb2d-cf1d-4921-8eb7-3c443e65c251 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:39.964129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768abb2d-cf1d-4921-8eb7-3c443e65c251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.087274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-768abb2d-cf1d-4921-8eb7-3c443e65c251 None None] MFA Rules not processed for user `b9a0adcfed5c4a8ba9366a1a69945e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:40.088594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-768abb2d-cf1d-4921-8eb7-3c443e65c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.096054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532fbfd-4bc3-4744-9e2d-21b57ebd59f3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.129288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.129288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.129288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.129288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.129590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.129590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.258125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 855/1700] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:39 2022] POST /identity/v3/projects => generated 379 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:40.263304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.280051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 846/1701] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:40.280051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c25e-3ac5-4ff2-971d-bbbb530264f7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.283012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c25e-3ac5-4ff2-971d-bbbb530264f7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.283012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c25e-3ac5-4ff2-971d-bbbb530264f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.350649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e624c25e-3ac5-4ff2-971d-bbbb530264f7 None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:40.352119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e624c25e-3ac5-4ff2-971d-bbbb530264f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.404280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59148594-a9cb-43dd-9cc2-819d17d60cc6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.418307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.419302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.419794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.420177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.421428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.423969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.434584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 847/1702] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:16:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:40.437300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.437587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.437863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.454663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 856/1703] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:40.455902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.471746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402c32c-8b06-483b-9368-d268bb32ccff admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.485215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.486776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.486776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.487578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:40.487697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.488469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.488828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.491497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.509097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 848/1704] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:16:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:40.509965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.511527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.589114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddd220-2c40-48e4-918c-5b0c193597a4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.593710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.594439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.594812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.595233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.617496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 857/1705] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:40 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:40.618450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.635357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0244ef-2cc5-4e66-bcbc-c0b9e7f9a949 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.640433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.641065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.641357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.641636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.642363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.644135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.660310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 858/1706] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:40.661283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.663257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.692058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb7531-17c2-4974-9b14-252073a79eb3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.706981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.707799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.708900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.708900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.726023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.728304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.731147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.732788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.744134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 859/1707] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:40.745365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.747167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:40.781272 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 849/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:40.782209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.799171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baaad9f-6e6d-46f7-b21c-bbc93fbf15b8 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.801081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507a7ef-d194-4cb1-9c7b-5ce7a1467427 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.810930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.810930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.810930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.810930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.816835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.817481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.817765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.818040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.828104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.829964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.841394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.849229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.861443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 850/1709] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:40.862371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.877939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 860/1710] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:40.879383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:40.903085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77e621-34de-485c-be52-eabe565f0b34 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.908433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefd788-963b-491b-9394-8cdf2e6a889b service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:40.921869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.922653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.922953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.923230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] PATH_INFO: `/v3/projects/da6dbee197ed470aaecc185c9ece1a21/users/e7ba5ad8a0f64c35807e6f17d70250ed/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.935841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:40.937528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:40.938552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:40.939004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] PATH_INFO: `/v3/projects/80f57f9a547849878cedf04010943e43/users/06ab52f2db98407dbe4ded671fb3bac2/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:40.969132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6dbee197ed470aaecc185c9ece1a21, user_id=e7ba5ad8a0f64c35807e6f17d70250ed, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.972040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb816c-4ae5-4ed5-8017-f9ddf5c61f03 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:40.982094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f57f9a547849878cedf04010943e43, user_id=06ab52f2db98407dbe4ded671fb3bac2, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:40.989294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:41.042513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:41.096550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 851/1711] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:40 2022] PUT /identity/v3/projects/da6dbee197ed470aaecc185c9ece1a21/users/e7ba5ad8a0f64c35807e6f17d70250ed/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:41.103185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:41.103477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:41.103782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:41.156697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:41.174522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 861/1712] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:40 2022] PUT /identity/v3/projects/80f57f9a547849878cedf04010943e43/users/06ab52f2db98407dbe4ded671fb3bac2/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:41.175716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:41.180781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:41.347761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754fd3a-a25a-4b83-92c8-fa1ca2dd7836 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:41.352942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 852/1713] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:41.367396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:41.375771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:41.376414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:41.376701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:41.377856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:41.378949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:41.381418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:41.430607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 862/1714] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:41.430607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:41.460351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e98a-bf85-4f8e-94e6-94d15edb1b50 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:41.471731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:41.472538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:41.472904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:41.473261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] PATH_INFO: `/v3/projects/80f57f9a547849878cedf04010943e43/users/06ab52f2db98407dbe4ded671fb3bac2/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:41.478010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f57f9a547849878cedf04010943e43, user_id=06ab52f2db98407dbe4ded671fb3bac2, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:41.486448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5222b9-617f-4f8a-9c77-9a7f98cd43b4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:41.515875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc6bb0-9b7b-4b1f-99dc-824ec30d93f2 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:41.533041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 863/1715] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:41 2022] PUT /identity/v3/projects/80f57f9a547849878cedf04010943e43/users/06ab52f2db98407dbe4ded671fb3bac2/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:41.546652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdca20-a23c-444a-a108-4686b5b85c51 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:41.546652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdca20-a23c-444a-a108-4686b5b85c51 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:41.546652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdca20-a23c-444a-a108-4686b5b85c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:41.627967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-66cdca20-a23c-444a-a108-4686b5b85c51 None None] MFA Rules not processed for user `06ab52f2db98407dbe4ded671fb3bac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:41.634562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-66cdca20-a23c-444a-a108-4686b5b85c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:41.650864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:41.651844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:41.652690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:41.654239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:41.779529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:41.780999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:41.794759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 853/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:41.882315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 864/1717] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:42.016076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:42.018459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:42.192025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa220e-827c-4899-a0ea-33bee7234de3 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:42.199418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:42.200509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:42.200925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:42.201784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:42.204211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:42.238837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:42.238941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:42.247532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d3aeb-1f58-4bc2-9e2b-11e408f72d8f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:42.248151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d3aeb-1f58-4bc2-9e2b-11e408f72d8f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:42.256269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d3aeb-1f58-4bc2-9e2b-11e408f72d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:42.256350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 854/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:42.271034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:42.321602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-518d3aeb-1f58-4bc2-9e2b-11e408f72d8f None None] MFA Rules not processed for user `1d22ea3fb8fa4bd99537fb83279b2d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:42.326084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-518d3aeb-1f58-4bc2-9e2b-11e408f72d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:42.422307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d179c-548c-4022-86f5-66f1b9944fc1 service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:42.440283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:42.441015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:42.442736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:42.442736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:42.462911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:42.467829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:42.491653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 865/1719] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:42.493843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 855/1720] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:42.773689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:43.137200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a298fa-4dc4-4cee-bc52-fb03ab44ce3e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:43.137830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a298fa-4dc4-4cee-bc52-fb03ab44ce3e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:43.138299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a298fa-4dc4-4cee-bc52-fb03ab44ce3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:43.272734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b4a298fa-4dc4-4cee-bc52-fb03ab44ce3e None None] MFA Rules not processed for user `8d154e96376a49918658b0c340c722e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:43.276313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b4a298fa-4dc4-4cee-bc52-fb03ab44ce3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:43.342776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:43.346105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:43.395478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d6fa4-9be1-48c6-aaac-9bec714a0f91 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:43.404843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:43.404843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:43.404843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:43.404843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] PATH_INFO: `/v3/users/b7aeee13c8a84f0aa29f9190680745c9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:43.418142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7aeee13c8a84f0aa29f9190680745c9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:43.424615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:43.435692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 866/1721] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:43.745056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7aeee13c8a84f0aa29f9190680745c9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:43.746576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7aeee13c8a84f0aa29f9190680745c9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:43.809403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7aeee13c8a84f0aa29f9190680745c9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:43.972564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7aeee13c8a84f0aa29f9190680745c9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:43.993222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7aeee13c8a84f0aa29f9190680745c9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:44.025458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 856/1722] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:43 2022] DELETE /identity/v3/users/b7aeee13c8a84f0aa29f9190680745c9 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:44.027432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:44.028691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:44.029187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:44.119835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:44.121556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:44.249830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:44.251988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:44.371350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 867/1723] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:44.396717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:44.400615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:44.474745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836c52d-6f2a-4c39-bb88-7f3fe04213c4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:44.503138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:44.503138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:44.503138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:44.503138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] PATH_INFO: `/v3/users/84d8b95368f2474e9193e166fcce977c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:44.511700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d8b95368f2474e9193e166fcce977c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:44.514517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:44.559500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f3ee9-49e1-4747-87d5-f88d6d70a98d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:44.572242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:44.572874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:44.573210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:44.573492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:44.624549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:44.626242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90c8ce-ec10-4048-99ed-a5fe0bbff103 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:44.682899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 868/1724] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:44.875379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d8b95368f2474e9193e166fcce977c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:44.878396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d8b95368f2474e9193e166fcce977c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:44.902649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d8b95368f2474e9193e166fcce977c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:45.006768 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d8b95368f2474e9193e166fcce977c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:45.019423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d8b95368f2474e9193e166fcce977c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:45.031509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 857/1725] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:44 2022] DELETE /identity/v3/users/84d8b95368f2474e9193e166fcce977c => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:45.036174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:45.036464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:45.036743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:45.065261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:45.066987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:45.235907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 869/1726] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:45.239672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:45.241924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:45.362782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98262ae-9f83-4bd1-855b-0054e4f9727d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:45.367503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:45.368149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:45.368437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:45.369027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:45.384793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:45.396659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2ba07-8f40-4c6c-90ce-41b92091d201 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:45.408869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 858/1727] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:45.782425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:45.784686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:45.819195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d72d3f-17ee-4267-82f4-098b7d779727 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:45.854925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:45.854925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:45.854925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:45.854925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] PATH_INFO: `/v3/projects/bdfe27a09bb6426a84d193f9dda3c43f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:45.857315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfe27a09bb6426a84d193f9dda3c43f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:45.862702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:45.868415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:45.868905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:45.869072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:45.932376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] MFA Rules not processed for user `e7ba5ad8a0f64c35807e6f17d70250ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:45.936085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.130958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 859/1728] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:46.132779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:46.136413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.151283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85cc75-45ef-492a-8adb-7f3e24616d2b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:46.156093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:46.156708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.156991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.157292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.157928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:46.162303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:46.285411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfe27a09bb6426a84d193f9dda3c43f', 'request_id': 'req-092eed56-6965-4854-b59c-f902555cdc5c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.295148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfe27a09bb6426a84d193f9dda3c43f', 'request_id': 'req-092eed56-6965-4854-b59c-f902555cdc5c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.407732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 860/1729] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:46 2022] POST /identity/v3/projects => generated 403 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:46.410071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:46.412190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.445839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfe27a09bb6426a84d193f9dda3c43f', 'request_id': 'req-092eed56-6965-4854-b59c-f902555cdc5c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.483806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] The token cache is being invalidate because project bdfe27a09bb6426a84d193f9dda3c43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:46.484154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.485473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.496107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 870/1730] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:45 2022] DELETE /identity/v3/projects/bdfe27a09bb6426a84d193f9dda3c43f => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:46.497256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:46.500909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.575221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2778d3-edb3-45a2-8f85-538b5790a6aa admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:46.581001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:46.581721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.582047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.582330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] PATH_INFO: `/v3/projects/92beeeda046c4ce88fb959efab32ea8c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.583571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92beeeda046c4ce88fb959efab32ea8c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:46.586560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:46.613864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eed56-6965-4854-b59c-f902555cdc5c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:46.626370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:46.627075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.630508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.630508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.630508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:46.630987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de07930-c944-4bda-b3a6-f9545352e2c8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:46.698874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:46.751142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 871/1731] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:46 2022] POST /identity/v3/users => generated 308 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:46.753495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.753858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.754145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.795768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:46.797001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.808215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92beeeda046c4ce88fb959efab32ea8c', 'request_id': 'req-7d278417-2ce2-4bd2-a129-a4cab582c97e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.920071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 872/1732] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:46.921244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:46.931673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65becd3c-c575-4682-ba79-03efff1ae6c7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:46.936382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:46.937036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.937328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.937764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.938266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:46.939975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f40813-0658-4761-be33-33d67e0db56b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:46.952674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 873/1733] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:46.954145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:46.954445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:46.954784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:46.979879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92beeeda046c4ce88fb959efab32ea8c', 'request_id': 'req-7d278417-2ce2-4bd2-a129-a4cab582c97e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:46.981823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:46.991158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:46.991300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92beeeda046c4ce88fb959efab32ea8c', 'request_id': 'req-7d278417-2ce2-4bd2-a129-a4cab582c97e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.001442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] The token cache is being invalidate because project 92beeeda046c4ce88fb959efab32ea8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:47.002103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.003154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.005420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 861/1734] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:46 2022] DELETE /identity/v3/projects/92beeeda046c4ce88fb959efab32ea8c => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:47.007148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.009432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:47.023104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 874/1735] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:47.024116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.026560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d278417-2ce2-4bd2-a129-a4cab582c97e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.029911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:47.044275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262529-294b-4a6c-994d-4c9cd8c7c7e9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.051789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.053268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.053450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.054569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] PATH_INFO: `/v3/users/06ab52f2db98407dbe4ded671fb3bac2` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.055504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.056481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.056849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.057354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] PATH_INFO: `/v3/projects/6289d9282a5d4136939b687e6902c906/users/422b2ae0f9e1466ba0be69b90f4ea1ad/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.057460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ab52f2db98407dbe4ded671fb3bac2)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.060523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.101583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6289d9282a5d4136939b687e6902c906, user_id=422b2ae0f9e1466ba0be69b90f4ea1ad, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.104446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.126690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:47.243834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 862/1736] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:47 2022] PUT /identity/v3/projects/6289d9282a5d4136939b687e6902c906/users/422b2ae0f9e1466ba0be69b90f4ea1ad/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:47.245261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.248074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:47.320671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672af5bd-e63b-4844-8bf1-9e8504679bd7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.325958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.326797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.327085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.327492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.328162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:47.328461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.330088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.355319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 863/1737] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:16:47.356247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.370032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf9536-cbd2-4013-82a2-27877e6bd308 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.377243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ab52f2db98407dbe4ded671fb3bac2'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.380448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ab52f2db98407dbe4ded671fb3bac2'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.380733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.380972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.382168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.382712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.383410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.385184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.396486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 864/1738] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:47.400290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.402157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:47.412812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ab52f2db98407dbe4ded671fb3bac2'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.416637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c662b78-41fa-4ade-92e9-e1a06b0e22d4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.423203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.423881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.424170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.424442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.425095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.427025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.539785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ab52f2db98407dbe4ded671fb3bac2'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.548197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ab52f2db98407dbe4ded671fb3bac2'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:47.555559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 875/1739] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:47 2022] DELETE /identity/v3/users/06ab52f2db98407dbe4ded671fb3bac2 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:47.556458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.557823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:47.615489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 865/1740] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:47 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:47.617926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:47.748427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a4fa7-b122-438f-bde9-da735ebf2dd9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.768856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2fcc7-1fcc-42de-9e95-836cfcf4a556 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:47.774967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.775506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.775842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.776196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.776889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.778903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.832116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:47.832796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:47.833082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:47.833363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] PATH_INFO: `/v3/projects/6289d9282a5d4136939b687e6902c906/users/422b2ae0f9e1466ba0be69b90f4ea1ad/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:47.836918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6289d9282a5d4136939b687e6902c906, user_id=422b2ae0f9e1466ba0be69b90f4ea1ad, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:47.840017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:47.852834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 876/1741] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:47 2022] PUT /identity/v3/projects/6289d9282a5d4136939b687e6902c906/users/422b2ae0f9e1466ba0be69b90f4ea1ad/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:47.970169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 866/1742] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:47 2022] POST /identity/v3/users => generated 287 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:16:47.974461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:48.164753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28c4df-2003-49db-92c5-916503bad951 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:48.174267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:48.176021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.176796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.177534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.179988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:48.184064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:48.198374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 877/1743] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:48.202356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:48.223238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b44c7-fa1c-43a3-9d61-6aa062e07a65 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:48.242379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:48.242379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.242379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.242379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.242770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:48.244205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbdeac-af97-46a5-8ac3-1700c9a58dcc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:48.276667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 867/1744] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:16:48.280059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:48.301500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340855d-ed83-441c-925e-86e816fa77d5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:48.322878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:48.323599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.326710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.326710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] PATH_INFO: `/v3/projects/1ae01bb200e649d0b23f41d09ab1b769/users/37dad0317eb546e2a0a5bad75f72c96f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.366811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae01bb200e649d0b23f41d09ab1b769, user_id=37dad0317eb546e2a0a5bad75f72c96f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:48.369919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:48.492475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 878/1745] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:48 2022] PUT /identity/v3/projects/1ae01bb200e649d0b23f41d09ab1b769/users/37dad0317eb546e2a0a5bad75f72c96f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:48.505595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.505595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.505886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.594532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] MFA Rules not processed for user `37dad0317eb546e2a0a5bad75f72c96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:48.595419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:48.674731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 868/1746] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:48.686446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:48.688317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:48.756566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:48.758494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:48.803800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dda19e-b53e-4d60-b155-9e08212541db admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:48.835575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:48.837245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.837535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.837810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.870609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:48.877369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee2706-d3a6-4dc6-9e73-5a27f16815b1 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:48.898317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 879/1747] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:48.960644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af560-3da7-4605-97d8-b65a0b3be0f7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:48.964818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:48.965814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:48.966100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:48.966375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] PATH_INFO: `/v3/projects/80f57f9a547849878cedf04010943e43` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:48.968652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f57f9a547849878cedf04010943e43)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:48.970708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:49.132693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c702-7d2f-4234-9a80-d08cb8707e85 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.133198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c702-7d2f-4234-9a80-d08cb8707e85 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.133622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40c702-7d2f-4234-9a80-d08cb8707e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.247334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f57f9a547849878cedf04010943e43', 'request_id': 'req-eee45429-3c25-4057-811d-d941b176002a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:49.258393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7e40c702-7d2f-4234-9a80-d08cb8707e85 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:49.265511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7e40c702-7d2f-4234-9a80-d08cb8707e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:49.385325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f57f9a547849878cedf04010943e43', 'request_id': 'req-eee45429-3c25-4057-811d-d941b176002a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:49.418212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f57f9a547849878cedf04010943e43', 'request_id': 'req-eee45429-3c25-4057-811d-d941b176002a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:49.442468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 880/1748] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:16:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:49.464202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] The token cache is being invalidate because project 80f57f9a547849878cedf04010943e43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:49.464202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:49.464202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eee45429-3c25-4057-811d-d941b176002a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:49.471622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 869/1749] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:48 2022] DELETE /identity/v3/projects/80f57f9a547849878cedf04010943e43 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:16:49.478848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.478848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.478848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.517585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:49.519281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:49.612757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 881/1750] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:49.614457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.614865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.615287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.646602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:49.648946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:49.674088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 870/1751] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:49.676742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:49.701255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:49.706925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:49.710509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619656-83f7-41d6-bea8-ec6d41521d13 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:49.727381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:49.728093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.728378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.728653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.729330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:49.729636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:49.735031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:49.748463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcc417-9efb-47a4-b892-b56c4b8baa78 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:49.774800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:49.782473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.783204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.783664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] PATH_INFO: `/v3/users/37dad0317eb546e2a0a5bad75f72c96f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.786395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37dad0317eb546e2a0a5bad75f72c96f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:49.790547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:49.814836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 882/1752] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:16:49.826306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:49.873152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fa96-9f26-4f3f-8d56-d363d4a239c5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:49.879276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:49.880296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:49.880854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:49.881342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:49.882453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:49.885175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:50.244042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 883/1753] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:49 2022] POST /identity/v3/projects => generated 383 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:50.251647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:50.393798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda72f4-0300-4158-b00a-a6ca7d09c2fa admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:50.399666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:50.400329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:50.400620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:50.400950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:50.401663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:50.403823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:50.614335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37dad0317eb546e2a0a5bad75f72c96f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:50.615135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37dad0317eb546e2a0a5bad75f72c96f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:50.634147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37dad0317eb546e2a0a5bad75f72c96f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:50.695750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 884/1754] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:50 2022] POST /identity/v3/users => generated 292 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:50.699541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:50.738974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37dad0317eb546e2a0a5bad75f72c96f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:50.745492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:50.755554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37dad0317eb546e2a0a5bad75f72c96f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:50.766285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 871/1755] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:49 2022] DELETE /identity/v3/users/37dad0317eb546e2a0a5bad75f72c96f => generated 0 bytes in 1065 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:50.767747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:50.770084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:50.906759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01640e-3c65-4273-8834-dde1e41f7457 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:50.910218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f7ec8-29d7-4294-8bba-d88fefa95fb1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:50.916453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:50.917621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:50.918234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:50.918743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:50.919791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:50.923041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf17c-69fc-48ea-bf4c-cfea64cafe05 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:50.938177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 872/1756] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:50.942175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:50.942954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:50.943370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:50.968255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:50.970092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:50.984920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 873/1757] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:50.988131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.001099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cbfd6-dc09-4f07-b64f-875b1b0ee798 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.007002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.008623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.009057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.009480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.010645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.013265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.030459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.031699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.031699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.031853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.033775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:51.036351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 874/1758] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:51.041858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.060120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f6497-3b2f-4f70-a050-a974b49e9434 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.066268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.067212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.067798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.069020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] PATH_INFO: `/v3/projects/e41a4a2401544ce28d7c7edc8c2f72b2/users/e9b9c005a9fa4e2e90e47b71d556f121/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.110452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41a4a2401544ce28d7c7edc8c2f72b2, user_id=e9b9c005a9fa4e2e90e47b71d556f121, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.113337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78b7f9-2952-4313-8b9d-36b937d76811 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.122134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.124885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.137078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 885/1759] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:16:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:16:51.140324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.141874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:51.297592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 875/1760] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:51 2022] PUT /identity/v3/projects/e41a4a2401544ce28d7c7edc8c2f72b2/users/e9b9c005a9fa4e2e90e47b71d556f121/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:51.307414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c7019-58de-40d1-9519-c70bb8f6206e service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.307968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdadb0-6f96-4554-ad11-8c84e6c37f2c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.308424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdadb0-6f96-4554-ad11-8c84e6c37f2c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.308715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdadb0-6f96-4554-ad11-8c84e6c37f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.370207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-aabdadb0-6f96-4554-ad11-8c84e6c37f2c None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:51.377664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-aabdadb0-6f96-4554-ad11-8c84e6c37f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:51.421165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.421943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.422305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.422701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.573047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.574533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.580114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 876/1761] 10.222.0.76 () {58 vars in 1015 bytes} [Fri Aug 26 06:16:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:51.582414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24fb0c-0fa4-40c5-9720-c94ab600c984 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.582935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24fb0c-0fa4-40c5-9720-c94ab600c984 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.584042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24fb0c-0fa4-40c5-9720-c94ab600c984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.618667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 886/1762] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:51.619827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.623712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:51.653049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb84d80-752c-471f-8fa6-c1cfe4433ee8 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.659527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.660297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.665953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.665953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.673239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1a24fb0c-0fa4-40c5-9720-c94ab600c984 None None] MFA Rules not processed for user `e9b9c005a9fa4e2e90e47b71d556f121`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:51.674539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1a24fb0c-0fa4-40c5-9720-c94ab600c984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:51.679765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.681248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.732997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 887/1763] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:51.772769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.788866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24568d82-75ae-4503-ae71-6f30ced20ca3 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.798458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.800142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.809226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.809226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.828472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.833504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:51.861462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 888/1764] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:51.942335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 877/1765] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:51.948815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:51.963301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d4685-0c9c-41a3-bf88-845ceeeaba9f service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:51.969201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:51.969824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:51.970322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:51.970658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:51.971565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:51.974153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:52.172737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:52.228663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:52.228955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:52.229326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:52.232682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 889/1766] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:51 2022] POST /identity/v3/projects => generated 383 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:52.237469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:52.303197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] MFA Rules not processed for user `422b2ae0f9e1466ba0be69b90f4ea1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:52.309978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:52.377927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:52.430070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd81-fc3d-4424-a63c-067d208240f5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:52.435108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:52.436086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:52.436395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:52.436660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:52.437305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:52.439495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:52.520731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 878/1767] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:52.604239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 890/1768] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:52 2022] POST /identity/v3/users => generated 298 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:52.609328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:52.644455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce0ba-e257-451f-8778-99409f179f98 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:52.664945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:52.665743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:52.666128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:52.666804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:52.667566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:52.669292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:52.718651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 879/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:52.719736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:52.736415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fac7e-220d-4305-9157-8c74b13054bf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:52.743341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:52.744421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:52.744756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:52.745163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] PATH_INFO: `/v3/projects/0ca44e552ada4164a77bb8911b0287ee/users/2cc1bbb5627e4d1190372046705fc5cc/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:52.784236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca44e552ada4164a77bb8911b0287ee, user_id=2cc1bbb5627e4d1190372046705fc5cc, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:52.793112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:53.342913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 891/1770] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:52 2022] PUT /identity/v3/projects/0ca44e552ada4164a77bb8911b0287ee/users/2cc1bbb5627e4d1190372046705fc5cc/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:53.383491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:53.392134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eff8c1-6001-4a81-8078-ef2f57c32d5e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:53.513115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:53.513831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:53.514402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:53.517331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:53.518774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:53.524051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcad8f9-3985-4225-89ec-e368da904125 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:53.554282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:53.554974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 880/1771] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:53.587017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe984d-b7a1-4a10-98d0-445f9b3a9d43 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:53.609004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:53.610099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:53.610485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:53.610977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] PATH_INFO: `/v3/projects/0ca44e552ada4164a77bb8911b0287ee/users/2cc1bbb5627e4d1190372046705fc5cc/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:53.613895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca44e552ada4164a77bb8911b0287ee, user_id=2cc1bbb5627e4d1190372046705fc5cc, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:53.621971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:53.660080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:53.663203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:53.663754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:53.684235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 892/1772] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:53 2022] PUT /identity/v3/projects/0ca44e552ada4164a77bb8911b0287ee/users/2cc1bbb5627e4d1190372046705fc5cc/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:53.753357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] MFA Rules not processed for user `2cc1bbb5627e4d1190372046705fc5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:53.753357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:53.868599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 881/1773] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:53.892991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:53.896661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:54.059869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681cf3-1733-4ceb-9080-28a1db079f20 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:54.067321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:54.069070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:54.069766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:54.070499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:54.119153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:54.121498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed9141-b67d-4770-b235-dea132c6ad30 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:54.150198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 893/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:54.380647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:54.401491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd66d-03f9-4b0c-81f7-5686005597aa None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:54.406880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:54.408346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:54.408823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:54.409288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:54.577251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:54.580700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3b5a4-f447-438a-beec-d64ba530cbe9 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:54.613987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 882/1775] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:54.983138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060e0d-9d40-41d2-972a-abdf3153ab0a None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:54.983138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060e0d-9d40-41d2-972a-abdf3153ab0a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:54.983138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49060e0d-9d40-41d2-972a-abdf3153ab0a None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:54.983975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 894/1776] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:55.024349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61803-e8bb-4948-b5c1-942d98bc30be None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.024349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61803-e8bb-4948-b5c1-942d98bc30be None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.024349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61803-e8bb-4948-b5c1-942d98bc30be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.065439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c5e61803-e8bb-4948-b5c1-942d98bc30be None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:55.066865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c5e61803-e8bb-4948-b5c1-942d98bc30be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:55.267873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 883/1777] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:16:55.357442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b61c0f-28af-4a94-b8bb-7419240aef41 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.358074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b61c0f-28af-4a94-b8bb-7419240aef41 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.358481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b61c0f-28af-4a94-b8bb-7419240aef41 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.361050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 895/1778] 10.222.0.40 () {54 vars in 961 bytes} [Fri Aug 26 06:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:55.367771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5abe-7d34-4622-9363-25b5b7bef894 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.368312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5abe-7d34-4622-9363-25b5b7bef894 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.368718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5abe-7d34-4622-9363-25b5b7bef894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.388263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:55.415650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-932a5abe-7d34-4622-9363-25b5b7bef894 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:55.417388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-932a5abe-7d34-4622-9363-25b5b7bef894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:55.439719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 884/1779] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Aug 26 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:55.458127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b61c0f-28af-4a94-b8bb-7419240aef41 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:55.479914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0308-c86c-4320-97de-3b29d25deaf2 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.480198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0308-c86c-4320-97de-3b29d25deaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.480707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f0308-c86c-4320-97de-3b29d25deaf2 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.482305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 885/1780] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:16:55.486135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b61c0f-28af-4a94-b8bb-7419240aef41 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:55.497733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:55.499475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.499475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.499475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.527607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:55.529251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:55.550386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 896/1781] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:55.594231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ff62-8dea-44d0-9765-c1ba6665d147 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:55.594451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ff62-8dea-44d0-9765-c1ba6665d147 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:55.594819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432ff62-8dea-44d0-9765-c1ba6665d147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:55.680829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f432ff62-8dea-44d0-9765-c1ba6665d147 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:55.683871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f432ff62-8dea-44d0-9765-c1ba6665d147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:55.972097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 886/1782] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:56.036707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:56.039892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.137030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2303-0a68-40fe-9a5a-41c29723ad69 service placement] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:56.154914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:56.156070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.156435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.156791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] PATH_INFO: `/v3/projects/1ae01bb200e649d0b23f41d09ab1b769` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.158197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae01bb200e649d0b23f41d09ab1b769)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:56.161073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:56.231299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.231866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.232364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.247577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 887/1783] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:16:56 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:16:56.310507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:56.314393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.317062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae01bb200e649d0b23f41d09ab1b769', 'request_id': 'req-2d281c40-ee9e-4074-9ef9-9a62263df797'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:56.325418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae01bb200e649d0b23f41d09ab1b769', 'request_id': 'req-2d281c40-ee9e-4074-9ef9-9a62263df797'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:56.477914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae01bb200e649d0b23f41d09ab1b769', 'request_id': 'req-2d281c40-ee9e-4074-9ef9-9a62263df797'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:56.482539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af098eb-0189-4638-ad85-8cac30a47c11 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:56.494449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:56.495139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.495532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.495807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.499887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] The token cache is being invalidate because project 1ae01bb200e649d0b23f41d09ab1b769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:16:56.500495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:56.502018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2d281c40-ee9e-4074-9ef9-9a62263df797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:56.504269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 897/1784] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:16:56 2022] DELETE /identity/v3/projects/1ae01bb200e649d0b23f41d09ab1b769 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:56.508283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:56.508283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0231a-099a-4e4b-95e9-a524ee000b7e service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:56.511960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.512579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.514763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.520243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 888/1785] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:16:56.522514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7996c1-9301-4aae-8c14-302e93adbf32 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.523015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7996c1-9301-4aae-8c14-302e93adbf32 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.523533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7996c1-9301-4aae-8c14-302e93adbf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.547497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:56.548979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.563270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5c7996c1-9301-4aae-8c14-302e93adbf32 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:56.567094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 898/1786] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:16:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:56.578807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5c7996c1-9301-4aae-8c14-302e93adbf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.657865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:56.662269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.719863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 889/1787] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:56.727060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.727698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.728075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.821617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:56.825004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:56.859510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 890/1788] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:56.878417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:56.896457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03d3d9-2c2a-4091-a011-763e3e89f30c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:56.902920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:56.903826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.904434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.904914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.905864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:16:56.906349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:56.908499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:56.931416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 891/1789] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:16:56.932275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:56.981420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853d89-121a-4ed0-92fe-0a0ebd5b48ff admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:56.989162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ff29-810a-4195-b901-34946110959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:56.990007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ff29-810a-4195-b901-34946110959c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:56.990998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ff29-810a-4195-b901-34946110959c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:56.992344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936ff29-810a-4195-b901-34946110959c admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:56.994950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936ff29-810a-4195-b901-34946110959c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:56.998455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936ff29-810a-4195-b901-34946110959c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:57.005570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e8d1-8dca-40f1-a1f5-b2ed356335f5 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:57.045192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:57.045842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:57.046357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:57.046357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] PATH_INFO: `/v3/users/e9b9c005a9fa4e2e90e47b71d556f121` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:57.058378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b9c005a9fa4e2e90e47b71d556f121)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:57.076254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:57.243381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 892/1790] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:56 2022] POST /identity/v3/projects => generated 363 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:16:57.247242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ff29-810a-4195-b901-34946110959c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:57.253380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e936ff29-810a-4195-b901-34946110959c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:57.426061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b9c005a9fa4e2e90e47b71d556f121'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:57.426845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b9c005a9fa4e2e90e47b71d556f121'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:57.441329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936ff29-810a-4195-b901-34946110959c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:57.462456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:57.463362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:57.463863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:57.464386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:57.466927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b9c005a9fa4e2e90e47b71d556f121'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:57.468910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:57.483267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:57.485749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:57.502510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 893/1791] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:57.504351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:57.506835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:57.596988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b9c005a9fa4e2e90e47b71d556f121'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:57.615173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-970ca763-bf76-4034-bdc9-b2f8ef888525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b9c005a9fa4e2e90e47b71d556f121'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:57.631803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 899/1792] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:56 2022] DELETE /identity/v3/users/e9b9c005a9fa4e2e90e47b71d556f121 => generated 0 bytes in 973 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:57.643971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:57.644282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:57.644545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:57.713818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cdab-ec18-44b0-ad8e-3cc2eb6db2b8 service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:57.739248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:57.742991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:57.929697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:57.933844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:57.935645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:57.937009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:57.942123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:57.943515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 900/1793] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:57.951134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:57.954958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:16:57.995365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:57.999290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:58.159323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 894/1794] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:57 2022] POST /identity/v3/users => generated 282 bytes in 656 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:58.160289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:58.177862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:58.186921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3325f-a56f-4e59-adbe-f080aaa11bd0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:58.194998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1b3c2-d879-4c2a-a8bc-bd0ef6dd134b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:58.196409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:58.198312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.199111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.199910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:58.200126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:58.201528 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.201816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.202097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] PATH_INFO: `/v3/users/e7ba5ad8a0f64c35807e6f17d70250ed` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:58.203398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ba5ad8a0f64c35807e6f17d70250ed)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:58.213412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:58.238744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:58.242160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:58.273480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 901/1795] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:16:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:16:58.276394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:58.296892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68726d-3ce9-4d08-b1ca-908087bcadb0 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:58.305217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:58.306129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.306605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.307184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:58.308576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:58.311248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:58.348359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 902/1796] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:58.353858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:58.377613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e7da5-36bf-4745-bfc8-5c8a16b13015 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:58.431467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:58.436337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.437632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.441982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:58.464456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:58.464456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:58.515630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 903/1797] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:16:58.538765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:58.573437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ba5ad8a0f64c35807e6f17d70250ed'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:58.574896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ba5ad8a0f64c35807e6f17d70250ed'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:58.622690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ba5ad8a0f64c35807e6f17d70250ed'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:58.627337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55378f8-daea-457b-8151-ea2a70ef2a9b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:58.660056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:58.662349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.665017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.666807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] PATH_INFO: `/v3/projects/61279822b1b74a578c2a2c380a3da6c4/users/a98d1f834dd0429ba9a24c9248061863/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:58.723520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61279822b1b74a578c2a2c380a3da6c4, user_id=a98d1f834dd0429ba9a24c9248061863, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:58.727428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:58.790327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ba5ad8a0f64c35807e6f17d70250ed'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:58.802061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-641b13d2-c9b9-47b5-837d-847d16721ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ba5ad8a0f64c35807e6f17d70250ed'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:16:58.813733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 895/1798] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:16:58 2022] DELETE /identity/v3/users/e7ba5ad8a0f64c35807e6f17d70250ed => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:58.970188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 904/1799] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:16:58 2022] PUT /identity/v3/projects/61279822b1b74a578c2a2c380a3da6c4/users/a98d1f834dd0429ba9a24c9248061863/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:16:58.975382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:58.976069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:58.976814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:59.068212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] MFA Rules not processed for user `a98d1f834dd0429ba9a24c9248061863`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:16:59.077198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:59.237575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 896/1800] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:16:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:16:59.243638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:59.245932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:16:59.410780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f1ad0-9a32-45fb-b553-431ac5ba2fe2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:16:59.436446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:16:59.437320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:16:59.437609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:16:59.437892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:16:59.438543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:16:59.441278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:16:59.615750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 905/1801] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:16:59 2022] POST /identity/v3/projects => generated 365 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:16:59.619914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:16:59.689692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4dd00-5af8-4718-840f-35b2cb0680a4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:00.063342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:00.063979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:00.064262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:00.064568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:00.065229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:00.067275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:00.296508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 897/1802] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:16:59 2022] POST /identity/v3/users => generated 289 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:00.301760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:00.329498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24a8c2-cf36-468c-85e6-411f0f833c19 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:00.343965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:00.344942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:00.345362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:00.345781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:00.346581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:00.348775 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:00.374806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 906/1803] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:00.379475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:00.405280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174592f4-ff12-4f1b-be38-832ec84ad4a4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:00.410648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:00.411405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:00.411749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:00.412091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] PATH_INFO: `/v3/projects/3d412157523a4625bd9f306327886922/users/ee320ea03f0d4fa8a648bd674b557555/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:00.422674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:00.425051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:00.451284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cafd43-d0e7-480d-8cb0-f6629aa297c3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:00.458836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d412157523a4625bd9f306327886922, user_id=ee320ea03f0d4fa8a648bd674b557555, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:00.460508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:00.480950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:00.481628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:00.481925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:00.482754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] PATH_INFO: `/v3/projects/da6dbee197ed470aaecc185c9ece1a21` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:00.484587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6dbee197ed470aaecc185c9ece1a21)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:00.486551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:00.583767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 898/1804] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:00 2022] PUT /identity/v3/projects/3d412157523a4625bd9f306327886922/users/ee320ea03f0d4fa8a648bd674b557555/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:00.586743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:00.752547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6dbee197ed470aaecc185c9ece1a21', 'request_id': 'req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:00.800961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6dbee197ed470aaecc185c9ece1a21', 'request_id': 'req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:00.847388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cffa2-6074-4298-9b9e-fbe3e2949655 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:00.949715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6dbee197ed470aaecc185c9ece1a21', 'request_id': 'req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:00.968882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] The token cache is being invalidate because project da6dbee197ed470aaecc185c9ece1a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:00.969346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:00.970131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:00.973846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 907/1805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:00 2022] DELETE /identity/v3/projects/da6dbee197ed470aaecc185c9ece1a21 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:00.974957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:00.985361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:01.044363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:01.045277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:01.045960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:01.046331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:01.061738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:01.064824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:01.093533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 899/1806] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:01.094704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:01.141707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:01.198695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79d568-7abf-466c-8bea-a14299fe4e1a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:01.211428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:01.219641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:01.222240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08aae4-8e9d-4d5e-b25b-e59446a7364a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:01.223014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:01.223014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] PATH_INFO: `/v3/users/422b2ae0f9e1466ba0be69b90f4ea1ad` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:01.227736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=422b2ae0f9e1466ba0be69b90f4ea1ad)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:01.233166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:01.233166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:01.233166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:01.233166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] PATH_INFO: `/v3/projects/e41a4a2401544ce28d7c7edc8c2f72b2` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:01.233604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41a4a2401544ce28d7c7edc8c2f72b2)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:01.233861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:01.236390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:01.379877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41a4a2401544ce28d7c7edc8c2f72b2', 'request_id': 'req-ff393f38-94f5-4988-96fb-b614dd61ea0d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.386599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:01.389931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41a4a2401544ce28d7c7edc8c2f72b2', 'request_id': 'req-ff393f38-94f5-4988-96fb-b614dd61ea0d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.482232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41a4a2401544ce28d7c7edc8c2f72b2', 'request_id': 'req-ff393f38-94f5-4988-96fb-b614dd61ea0d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.509795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] The token cache is being invalidate because project e41a4a2401544ce28d7c7edc8c2f72b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:01.510261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.515681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.517956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 908/1807] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:00 2022] DELETE /identity/v3/projects/e41a4a2401544ce28d7c7edc8c2f72b2 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:01.521478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:01.536017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:01.603609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422b2ae0f9e1466ba0be69b90f4ea1ad'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.607233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422b2ae0f9e1466ba0be69b90f4ea1ad'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.651042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422b2ae0f9e1466ba0be69b90f4ea1ad'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.750399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422b2ae0f9e1466ba0be69b90f4ea1ad'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.762106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff393f38-94f5-4988-96fb-b614dd61ea0d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:01.767409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422b2ae0f9e1466ba0be69b90f4ea1ad'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:01.792813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:01.795936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:01.795936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:01.795936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] PATH_INFO: `/v3/projects/3d412157523a4625bd9f306327886922/users/ee320ea03f0d4fa8a648bd674b557555/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:01.805485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d412157523a4625bd9f306327886922, user_id=ee320ea03f0d4fa8a648bd674b557555, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:01.806330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 900/1808] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:01 2022] DELETE /identity/v3/users/422b2ae0f9e1466ba0be69b90f4ea1ad => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:01.807442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:01.809692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:01.817839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:01.854884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 909/1809] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:01 2022] PUT /identity/v3/projects/3d412157523a4625bd9f306327886922/users/ee320ea03f0d4fa8a648bd674b557555/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:01.857045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:01.860520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:01.973825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda08a8-cfde-4937-9a17-da6f939b1787 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:02.077291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cae8d-6c90-4bb4-a211-d5e0fa7e21fd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:02.088976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:02.092802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:02.093499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:02.094143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:02.094842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] PATH_INFO: `/v3/users/2cc1bbb5627e4d1190372046705fc5cc` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:02.101256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc1bbb5627e4d1190372046705fc5cc)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:02.108208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:02.170095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:02.171434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:02.171801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:02.172234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] PATH_INFO: `/v3/users/b9a0adcfed5c4a8ba9366a1a69945e0c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:02.185307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a0adcfed5c4a8ba9366a1a69945e0c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:02.187114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:02.424155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc1bbb5627e4d1190372046705fc5cc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.426748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc1bbb5627e4d1190372046705fc5cc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.445364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a0adcfed5c4a8ba9366a1a69945e0c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.447599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc1bbb5627e4d1190372046705fc5cc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.447733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a0adcfed5c4a8ba9366a1a69945e0c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.493108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a0adcfed5c4a8ba9366a1a69945e0c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.593467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc1bbb5627e4d1190372046705fc5cc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.606477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a8208e98-a126-4b50-b02d-dbc78562f1c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc1bbb5627e4d1190372046705fc5cc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.625176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 910/1810] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:01 2022] DELETE /identity/v3/users/2cc1bbb5627e4d1190372046705fc5cc => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:02.632509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb5e6-609e-484c-b360-8ad0080a5ee4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:02.633786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb5e6-609e-484c-b360-8ad0080a5ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:02.634091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb5e6-609e-484c-b360-8ad0080a5ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:02.686674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a0adcfed5c4a8ba9366a1a69945e0c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.691356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a0adcfed5c4a8ba9366a1a69945e0c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:02.709126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 901/1811] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:01 2022] DELETE /identity/v3/users/b9a0adcfed5c4a8ba9366a1a69945e0c => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:02.713519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:02.716026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:02.729529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fedfb5e6-609e-484c-b360-8ad0080a5ee4 None None] MFA Rules not processed for user `ee320ea03f0d4fa8a648bd674b557555`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:02.731677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fedfb5e6-609e-484c-b360-8ad0080a5ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:02.911281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019660f8-7c28-43fa-ad89-ccc14c3a9928 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:02.929429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 911/1812] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:02.934234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:02.934234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:02.934424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:02.934512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:02.934512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] PATH_INFO: `/v3/projects/6289d9282a5d4136939b687e6902c906` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:02.934675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:02.934675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:02.934804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6289d9282a5d4136939b687e6902c906)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:02.938731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:02.965310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:02.966779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:02.980213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 912/1813] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:02.980909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:02.982850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:03.111175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb20e5-de39-4fad-b649-a2f88bb81bc5 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:03.120157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:03.121569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.122144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.123160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.145245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:03.148830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:03.155814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6289d9282a5d4136939b687e6902c906', 'request_id': 'req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:03.171890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 913/1814] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:03.173879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:03.176710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:03.285456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6289d9282a5d4136939b687e6902c906', 'request_id': 'req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:03.296913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6289d9282a5d4136939b687e6902c906', 'request_id': 'req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:03.327713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024e7a5-0431-4316-8684-b238330e0036 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:03.349172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] The token cache is being invalidate because project 6289d9282a5d4136939b687e6902c906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:03.349622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:03.350972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97f14f2f-6b74-4c70-9bd9-da2d87e3d02e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:03.356862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 902/1815] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:02 2022] DELETE /identity/v3/projects/6289d9282a5d4136939b687e6902c906 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:03.363046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae574a3-c033-469e-8768-710b0500e5cf None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.364160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae574a3-c033-469e-8768-710b0500e5cf None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.364397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae574a3-c033-469e-8768-710b0500e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.406094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1ae574a3-c033-469e-8768-710b0500e5cf None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:03.407870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1ae574a3-c033-469e-8768-710b0500e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:03.480482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:03.481858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.482517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.483155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.486752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:03.593813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 903/1816] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:03.595707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.604421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:03.606830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.607043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:03.608689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.631978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 914/1817] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:03.662484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:03.664281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:03.695015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 904/1818] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:03.698721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:03.710904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f5424-9c6f-4cc6-b545-f9bab5f0cf67 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:03.721155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:03.722140 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.722548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.722872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.724147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:03.724511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:03.726913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:03.747944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 915/1819] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:03.759058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:03.791257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6274bb-ffd6-49c1-a1ce-e4bf435f75e7 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:03.804809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:03.805448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:03.805733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:03.806037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:03.806802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:03.816253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.010815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 905/1820] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:03 2022] POST /identity/v3/projects => generated 363 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:04.015123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.059446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d50108-3538-4c12-9c4c-afb32891b59e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:04.206748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.214158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:04.255858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:04.257265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:04.263231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:04.263231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:04.263810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:04.265813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.469620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa1a3a-ded3-4556-b693-49a395b6710f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:04.486084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:04.486084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:04.486084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:04.486084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] PATH_INFO: `/v3/projects/0ca44e552ada4164a77bb8911b0287ee` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:04.486365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca44e552ada4164a77bb8911b0287ee)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:04.492922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.561082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 916/1821] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:04 2022] POST /identity/v3/users => generated 282 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:17:04.567683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.589996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe551a-fa37-40d6-acb4-16839678d0e9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:04.597674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:04.599030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:04.599692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:04.601606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:04.602711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:04.605808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.624057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 917/1822] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:04.630037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.643540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fe9dc-caa5-4dbf-9075-1679c939252a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:04.675094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:04.675094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:04.675094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:04.675094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:04.675574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:04.675574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.675574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 918/1823] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:04.682221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.701473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448451-acf6-4bab-9df2-cac88746f604 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:04.716239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:04.717274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:04.717750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:04.718152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] PATH_INFO: `/v3/projects/e24c359a18a54b3dbd3ccc2b2d79c20f/users/5f2e7d35221e474ba4319ccfeb7704d1/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:04.748160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24c359a18a54b3dbd3ccc2b2d79c20f, user_id=5f2e7d35221e474ba4319ccfeb7704d1, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:04.748160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:04.876715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:04.902514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 919/1824] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:04 2022] PUT /identity/v3/projects/e24c359a18a54b3dbd3ccc2b2d79c20f/users/5f2e7d35221e474ba4319ccfeb7704d1/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:04.904445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:04.923643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:04.955129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca44e552ada4164a77bb8911b0287ee', 'request_id': 'req-eb50f0ab-d1dd-45e7-8267-77dd60a87823'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.031025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a6b9-eec1-4f2d-8584-51cf43a8169c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:05.119327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:05.120611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.125713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.125713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] PATH_INFO: `/v3/projects/bb6fd7ac40454d578844c82354011594` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.125713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb6fd7ac40454d578844c82354011594)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:05.127199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:05.133233 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca44e552ada4164a77bb8911b0287ee', 'request_id': 'req-eb50f0ab-d1dd-45e7-8267-77dd60a87823'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.142300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca44e552ada4164a77bb8911b0287ee', 'request_id': 'req-eb50f0ab-d1dd-45e7-8267-77dd60a87823'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.164799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] The token cache is being invalidate because project 0ca44e552ada4164a77bb8911b0287ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:05.165257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.166399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-eb50f0ab-d1dd-45e7-8267-77dd60a87823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.169285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 906/1825] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:04 2022] DELETE /identity/v3/projects/0ca44e552ada4164a77bb8911b0287ee => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:05.171724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.172210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.172679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.208155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] MFA Rules not processed for user `5f2e7d35221e474ba4319ccfeb7704d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:05.209637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.316085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 907/1826] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:05.316085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:05.316085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.333487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66d971-507f-4415-87f8-12beb7932992 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:05.338658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb6fd7ac40454d578844c82354011594', 'request_id': 'req-efa4b46d-378a-4be8-89c8-3046cf1b2762'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.342400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb6fd7ac40454d578844c82354011594', 'request_id': 'req-efa4b46d-378a-4be8-89c8-3046cf1b2762'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.403239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:05.403887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.404816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.405106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] PATH_INFO: `/v3/users/a98d1f834dd0429ba9a24c9248061863` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.410789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98d1f834dd0429ba9a24c9248061863)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:05.413605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:05.430123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb6fd7ac40454d578844c82354011594', 'request_id': 'req-efa4b46d-378a-4be8-89c8-3046cf1b2762'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.447207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] The token cache is being invalidate because project bb6fd7ac40454d578844c82354011594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:05.448896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:05.449540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.450867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-efa4b46d-378a-4be8-89c8-3046cf1b2762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.456871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 920/1827] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:04 2022] DELETE /identity/v3/projects/bb6fd7ac40454d578844c82354011594 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:05.456871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.456871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.456871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.482339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:05.483783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.493443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 921/1828] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:05.495191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:05.503398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.508787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265cf21-d862-4592-a060-14d1f3cc951b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:05.513698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:05.514325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.514634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.515419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.515586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:05.517906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3972915f-382f-4d2a-bd11-c9c05e8cf1de admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:05.644213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 922/1829] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:05.645696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c38e5-71ce-46cd-a8a9-f20bcfea8e85 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.646027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c38e5-71ce-46cd-a8a9-f20bcfea8e85 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.646312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c38e5-71ce-46cd-a8a9-f20bcfea8e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.668731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e90c38e5-71ce-46cd-a8a9-f20bcfea8e85 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:05.669955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e90c38e5-71ce-46cd-a8a9-f20bcfea8e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.736311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 923/1830] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:05.737746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.738037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.738315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.769042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:05.769335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:05.777789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 924/1831] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:05.778649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:05.788764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56a24a-b363-4ecb-b9a1-5273d58b3e9e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:05.796016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:05.798670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98d1f834dd0429ba9a24c9248061863'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.799703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98d1f834dd0429ba9a24c9248061863'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:05.806258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:05.806258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:05.806258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:05.806258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:05.808247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac4f3-5008-4fe3-b188-b2eb315fd22c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:05.815106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98d1f834dd0429ba9a24c9248061863'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:06.058583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98d1f834dd0429ba9a24c9248061863'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:06.065979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 925/1832] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:05 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:06.067211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98d1f834dd0429ba9a24c9248061863'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:06.067658 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.068012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.068367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.076662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 908/1833] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:05 2022] DELETE /identity/v3/users/a98d1f834dd0429ba9a24c9248061863 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:06.078016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.080883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:06.095162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:06.096472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:06.160912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07119327-d37d-4abe-9e77-691e4e8cdac4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.166136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.167049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.167327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.167598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.168258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:06.168559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.171102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761291a0-735d-48d5-b7c9-8aa77b9fcc35 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.187325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 926/1834] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:06.188013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.189315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:06.193025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 909/1835] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:06.194700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.195188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.195644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.201405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7cfd8-199f-4b11-94d0-358c780e815e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.208700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.210138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.210513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.211018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.211628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.213301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.218266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:06.219753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:06.228302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 910/1836] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:06.232138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 927/1837] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:06.232399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.232510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.244611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69120f-b044-4f16-b8be-6b834cd0edcd None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.249656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.250188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee4185-a8ba-497b-9656-b37554e4d198 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.250462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.250606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.253126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.253126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:06.253126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.253546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.255946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.257740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.258027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.258308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.258989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.261113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.267171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 911/1838] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:06.268053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.280637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450fdf6-d5c1-4651-938f-7a20debf9cd9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.285816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.286495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.286823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.287100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] PATH_INFO: `/v3/projects/21d575f745694550acd890029cf13436/users/f3fec9e5844d471f8060ee61934de0b0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.311245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d575f745694550acd890029cf13436, user_id=f3fec9e5844d471f8060ee61934de0b0, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.314011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.418444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 928/1839] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:06.419285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.420952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:06.452474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 912/1840] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:06 2022] PUT /identity/v3/projects/21d575f745694550acd890029cf13436/users/f3fec9e5844d471f8060ee61934de0b0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:06.453956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.502387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481b138-9fda-4e13-9ba6-1a34a3e46c5d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.561276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0774da6-7448-493f-b7b1-3782a1bd7102 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.570517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.571301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.571663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.572016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.572827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.574925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.594564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.595230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.595521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.595960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.606800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.608291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.620563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 929/1841] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:06.621507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.634818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d757922-4087-46aa-a208-15488a55e599 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.639985 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.640440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.640772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.641478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.642072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.644142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.799431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 913/1842] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/projects => generated 385 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:06.800599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.838514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 930/1843] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:06.840180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.851039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a1618-6573-4cff-b14c-6f71e4c3600f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.879232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87960c2-7a20-4bb7-a995-f39733504f65 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.895815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.896975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.897562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.898170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.900231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.902963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.917587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 914/1844] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:06.918540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.929093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484747fc-9d78-495d-9552-c53a95b3e1dc admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.940625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.940625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.940969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.941168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.942193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.944990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.960186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 915/1845] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:06.962908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:06.977175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.986759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.986759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.986759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.986759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.987723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407545-a563-49c7-a88a-28b0a82e2b1a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:06.988163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca1ec1-3aa1-404d-a868-47dd02c382dc admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:06.988297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:06.988297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:06.988297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:06.988297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] PATH_INFO: `/v3/projects/21d575f745694550acd890029cf13436/users/f3fec9e5844d471f8060ee61934de0b0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:06.990664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d575f745694550acd890029cf13436, user_id=f3fec9e5844d471f8060ee61934de0b0, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:06.994801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.025058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 916/1846] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:06 2022] PUT /identity/v3/projects/21d575f745694550acd890029cf13436/users/f3fec9e5844d471f8060ee61934de0b0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:07.025058 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.127146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 931/1847] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:06 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:07.128697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.128989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.129265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.168193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031d0693-d650-4883-bf46-83b80ff0c758 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.171615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] MFA Rules not processed for user `f3fec9e5844d471f8060ee61934de0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:07.174024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:07.182525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.182525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.182525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.182525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.183065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.183065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.194001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 917/1848] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:07.195273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.208194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6e62c-27d3-4fda-81ba-d9d8c56960c4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.223676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.224602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.224602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.224870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.225828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.227524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.231440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:07.255776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 918/1849] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:07.257577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.275589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ffeb6-5f61-4c12-8059-df2e456cfc6f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.282479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.283359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.283655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.283920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] PATH_INFO: `/v3/projects/5b8dffc1266c42ad9369571561a715fd/users/49331867358f4bf08e4ea7b25c49ad2f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.323783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8dffc1266c42ad9369571561a715fd, user_id=49331867358f4bf08e4ea7b25c49ad2f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.331029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.368806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 932/1850] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:07.369691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.386941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ef78-a6c1-4f97-9765-18e64574b3ea None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.392076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.392815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.393261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.393722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.394503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.396357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.413611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 933/1851] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:07.414993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.417486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:07.495396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9083d96-e821-4f1d-92b2-4b0c88b46aa8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.498104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 919/1852] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:07 2022] PUT /identity/v3/projects/5b8dffc1266c42ad9369571561a715fd/users/49331867358f4bf08e4ea7b25c49ad2f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:07.499594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.527038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:07.581433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.582071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.582354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.582663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.600949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dad91d-d1a9-4071-b9ea-9a84e08e4820 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.607579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.608700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.609229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.609744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] PATH_INFO: `/v3/projects/c348892002ae4e3d81f8737ad8ce6aa2/users/a44102a2fcce4c3999860e15dfb6a6b5/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.634784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c348892002ae4e3d81f8737ad8ce6aa2, user_id=a44102a2fcce4c3999860e15dfb6a6b5, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.637511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.668047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.669528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664299-8327-435a-a0a0-a6ce1cf720b1 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.686730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 934/1853] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:07.688914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ff4a-fb60-447c-81b6-7fd116cd41f7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.689971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ff4a-fb60-447c-81b6-7fd116cd41f7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.690467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4ff4a-fb60-447c-81b6-7fd116cd41f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.737119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-87d4ff4a-fb60-447c-81b6-7fd116cd41f7 None None] MFA Rules not processed for user `49331867358f4bf08e4ea7b25c49ad2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:07.738527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-87d4ff4a-fb60-447c-81b6-7fd116cd41f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:07.743245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 920/1854] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:07 2022] PUT /identity/v3/projects/c348892002ae4e3d81f8737ad8ce6aa2/users/a44102a2fcce4c3999860e15dfb6a6b5/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:07.751730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:07.753875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:07.833212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 935/1855] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:07.835110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.835407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.835688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.873873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] MFA Rules not processed for user `a44102a2fcce4c3999860e15dfb6a6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:07.875134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:07.890212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b9ffb-cc58-459f-a399-4b20a6c492d3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:07.895352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:07.896083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:07.896349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:07.897175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] PATH_INFO: `/v3/projects/61279822b1b74a578c2a2c380a3da6c4` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:07.899622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61279822b1b74a578c2a2c380a3da6c4)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:07.903131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:07.984484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 936/1856] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:07.986554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.036788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61279822b1b74a578c2a2c380a3da6c4', 'request_id': 'req-6cde5882-b6ae-4bdd-b896-a992dee662b6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.055442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0825-328c-4b69-a711-862c179207fa None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.109549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.110162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.110449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.110783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.155389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61279822b1b74a578c2a2c380a3da6c4', 'request_id': 'req-6cde5882-b6ae-4bdd-b896-a992dee662b6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.165346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61279822b1b74a578c2a2c380a3da6c4', 'request_id': 'req-6cde5882-b6ae-4bdd-b896-a992dee662b6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.184750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] The token cache is being invalidate because project 61279822b1b74a578c2a2c380a3da6c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:08.185044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.185735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.187567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.187809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 921/1857] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:07 2022] DELETE /identity/v3/projects/61279822b1b74a578c2a2c380a3da6c4 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:08.192018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.192128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.192228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:08.204203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 937/1858] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:08.205287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.208189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:08.224055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0d439-fb2a-4958-be95-2c8969c6a947 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.230126 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.230967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.232042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.232042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.233943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:08.284662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde5882-b6ae-4bdd-b896-a992dee662b6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.292279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.293164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.293563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.294015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.294920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.297722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.326241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.327866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.341061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 938/1859] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:08.341925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.343657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:08.359764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b20c3-1839-4633-9495-3b40176eace1 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.372194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.372976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.373378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.373786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] PATH_INFO: `/v3/users/ee320ea03f0d4fa8a648bd674b557555` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.375238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee320ea03f0d4fa8a648bd674b557555)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.377667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.450683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 922/1860] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:08 2022] POST /identity/v3/projects => generated 387 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:08.455927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.471962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:08.566074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d4d1-7881-4591-ac57-3cc3d0bde683 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.578579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.578579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.578579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.578579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.579981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.579981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.655235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee320ea03f0d4fa8a648bd674b557555'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.656079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee320ea03f0d4fa8a648bd674b557555'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.674237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee320ea03f0d4fa8a648bd674b557555'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.739971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 923/1861] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:08 2022] POST /identity/v3/users => generated 294 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:08.744506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.746195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:08.760070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22268e-7ecd-43f4-8454-cc54bfb84750 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.765360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.767218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.767927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.768884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.772551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.776107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.793257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 924/1862] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:08.793257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.807620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0f6bc-2743-40f3-90d3-1c208eef5591 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.812720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.813350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.813640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.813906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.814078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee320ea03f0d4fa8a648bd674b557555'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.814731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.816645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:08.824705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-03f7dc1f-1add-4829-af2c-f980ed6401f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee320ea03f0d4fa8a648bd674b557555'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:08.831620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 925/1863] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:08.835974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:08.836914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 939/1864] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:08 2022] DELETE /identity/v3/users/ee320ea03f0d4fa8a648bd674b557555 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:08.846534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa6b5a-dee8-4e96-93c8-cd64e39d6b21 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:08.936932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:08.937622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:08.937921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:08.938191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] PATH_INFO: `/v3/projects/1d1fff03fa7240b89e06f388faf280e2/users/43b0994a9fcf4062acbc4435552e71d0/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:08.955799 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1fff03fa7240b89e06f388faf280e2, user_id=43b0994a9fcf4062acbc4435552e71d0, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:08.958728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.111330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 926/1865] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:08 2022] PUT /identity/v3/projects/1d1fff03fa7240b89e06f388faf280e2/users/43b0994a9fcf4062acbc4435552e71d0/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:09.126313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.127226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.128154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.161565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] MFA Rules not processed for user `43b0994a9fcf4062acbc4435552e71d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:09.163714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:09.251817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 940/1866] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:09.257009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:09.352217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46b0d-42be-4da7-83d4-8f40fa79d748 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:09.357236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:09.357867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.358143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.358447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.359220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:09.361168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.539097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 927/1867] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:09 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:09.543998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:09.559929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b15446-4e45-42fe-8a02-c457c19ca6b3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:09.627382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:09.630313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:09.636014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:09.636641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.636934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.637209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.637857 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:09.639846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.656734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730043b4-63f9-4dd9-b30f-2cf5b1a5bc5c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:09.661647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:09.663306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.663611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.663906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] PATH_INFO: `/v3/projects/3d412157523a4625bd9f306327886922` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.665396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d412157523a4625bd9f306327886922)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:09.669164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.752512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 941/1868] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:09 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:09.757471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:09.770404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c32e3-9da5-4960-b22c-37e3f778e200 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:09.776888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:09.777740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.778203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.778639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.780668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:09.781949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.809622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 942/1869] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:09.811664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:09.829650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d412157523a4625bd9f306327886922', 'request_id': 'req-8761e6ed-bb42-460e-8eab-a7805403f3da'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:09.831482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bef43-281f-499b-bbaa-3f40a4129aae admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:09.929829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:09.930513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.930851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.931134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] PATH_INFO: `/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19/users/d4c4fda878bc4dd2a5dd43f2fdb63569/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:09.948819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41e8c1ff5514a2699fc2cfda3d89a19, user_id=d4c4fda878bc4dd2a5dd43f2fdb63569, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:09.951972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:09.966612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d412157523a4625bd9f306327886922', 'request_id': 'req-8761e6ed-bb42-460e-8eab-a7805403f3da'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:09.973347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d412157523a4625bd9f306327886922', 'request_id': 'req-8761e6ed-bb42-460e-8eab-a7805403f3da'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:09.986821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] The token cache is being invalidate because project 3d412157523a4625bd9f306327886922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:09.987095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:09.987801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8761e6ed-bb42-460e-8eab-a7805403f3da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:09.989786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 928/1870] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:09 2022] DELETE /identity/v3/projects/3d412157523a4625bd9f306327886922 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:09.995869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a04c2b-3785-4832-bcee-c697ad763d2e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:09.996201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a04c2b-3785-4832-bcee-c697ad763d2e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:09.996524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a04c2b-3785-4832-bcee-c697ad763d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.018839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-01a04c2b-3785-4832-bcee-c697ad763d2e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:10.020139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-01a04c2b-3785-4832-bcee-c697ad763d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:10.030848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 929/1871] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:10.040108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.040537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.040965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.063340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:10.064582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:10.074216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 930/1872] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:10.078637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.090346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959ee93-4509-4d42-ba16-c8afeea7e400 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.096234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.097131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.097825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.098400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.099581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:10.101846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.103807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.108905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 943/1873] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:09 2022] PUT /identity/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19/users/d4c4fda878bc4dd2a5dd43f2fdb63569/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:10.117749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.120544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:10.122700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 931/1874] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:17:10.129191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.143860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7532e-5502-4474-a61b-b521a77135de admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.206243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86693d54-a7c0-4b2a-bf7e-35d77602ec6d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.213648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.214466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.214867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.215218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.216034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.220752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.255543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.265368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.265368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.265368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.265611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 944/1875] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:10.265699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.265699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.265844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.277904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd46256-63c1-4cc7-8d61-17801eb892f4 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.284233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.285085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.285477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.286093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] PATH_INFO: `/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19/users/d4c4fda878bc4dd2a5dd43f2fdb63569/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.288655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41e8c1ff5514a2699fc2cfda3d89a19, user_id=d4c4fda878bc4dd2a5dd43f2fdb63569, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.293039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba603-57a5-42e2-abe2-60492d80d6ee admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.297969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:10.310851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 945/1876] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:10 2022] PUT /identity/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19/users/d4c4fda878bc4dd2a5dd43f2fdb63569/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:10.321222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.321509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.321790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.361923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] MFA Rules not processed for user `d4c4fda878bc4dd2a5dd43f2fdb63569`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:10.363189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:10.388687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 932/1877] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:10 2022] POST /identity/v3/projects => generated 365 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:10.397993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.482092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 946/1878] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:10.485571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328d3d7-666d-4605-96e7-8066086b7fdf admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.491460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.492153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.492411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.492656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.493315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.495335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.521162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.523553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:10.596317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01cfd4-0db4-4a28-8d10-95debf1ea136 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.651366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.652006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.652291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.652576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.670773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 933/1879] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:10 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:17:10.672442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.686828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedbaa0e-b47b-4de7-ba15-2e96984fff62 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.698891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.698891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.698891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.698891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.699614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.701607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.717339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 934/1880] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:10.722048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.736935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.739132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.739351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7b42-79d9-4ecc-9b67-3c11a8d36554 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.747459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.748105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.748392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.748684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.749358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.750798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 947/1881] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:10.751108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5bf46-b888-423a-9f1b-3d285b4c42d2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.773172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 935/1882] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:10.774543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:10.789220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11436592-e4e4-4a3d-84d9-5f79fee435b0 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:10.794589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:10.795309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.795595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.795876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] PATH_INFO: `/v3/projects/74820d32d28e4657ae674b3c1ab7a32b/users/0b5ed82057184a0ea2c2453a346a4900/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:10.819218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74820d32d28e4657ae674b3c1ab7a32b, user_id=0b5ed82057184a0ea2c2453a346a4900, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:10.823502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:10.982949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 948/1883] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:10 2022] PUT /identity/v3/projects/74820d32d28e4657ae674b3c1ab7a32b/users/0b5ed82057184a0ea2c2453a346a4900/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:10.994960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:10.995413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:10.995858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:11.032125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] MFA Rules not processed for user `0b5ed82057184a0ea2c2453a346a4900`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:11.033390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:11.142749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 936/1884] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:11.150087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:11.294729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda722bf-9c71-4bc0-b390-417ccc2b3424 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:11.302130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:11.303181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:11.303651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:11.304178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:11.305011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:11.307794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:11.480166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 949/1885] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:11 2022] POST /identity/v3/projects => generated 367 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:11.485664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:11.510995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8890e-fca7-4165-a503-bcf8d98b26ae None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:11.608990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:11.609796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:11.610166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:11.610524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:11.611601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:11.613816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:11.731054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 937/1886] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:11 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:17:11.735732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:11.747731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb2037-e549-416d-8fab-a81dd7f3f12f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:11.756310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:11.761335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:11.761335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:11.761335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:11.761335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:11.762477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:11.777479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 950/1887] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:11.781911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:11.797221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1927d8f-793c-42c9-802c-e9075c1632b6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:11.802509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:11.803340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:11.803847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:11.804153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] PATH_INFO: `/v3/projects/e335343f5ba54c57b093091b3d3a2e49/users/2ef43401f0124d4dab2858f735c37a47/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:11.822891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e335343f5ba54c57b093091b3d3a2e49, user_id=2ef43401f0124d4dab2858f735c37a47, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:11.825583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:11.971901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 938/1888] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:11 2022] PUT /identity/v3/projects/e335343f5ba54c57b093091b3d3a2e49/users/2ef43401f0124d4dab2858f735c37a47/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:11.978272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:11.989250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d80007-b454-463d-aa5a-a25af70d005d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:12.058975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:12.059612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.059862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.060150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.060803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:12.062453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ffb8b-0560-4d4c-92bc-5d7cf8ef3fa2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:12.075939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 951/1889] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:12.080419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:12.095292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3cfe-e8eb-46bc-91b2-930ca5ad1de7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:12.101503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:12.102716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.103249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.103560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] PATH_INFO: `/v3/projects/e335343f5ba54c57b093091b3d3a2e49/users/2ef43401f0124d4dab2858f735c37a47/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.105767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e335343f5ba54c57b093091b3d3a2e49, user_id=2ef43401f0124d4dab2858f735c37a47, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:12.108830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:12.121294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 939/1890] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:12 2022] PUT /identity/v3/projects/e335343f5ba54c57b093091b3d3a2e49/users/2ef43401f0124d4dab2858f735c37a47/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:12.133991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.134285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.134569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.168051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] MFA Rules not processed for user `2ef43401f0124d4dab2858f735c37a47`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:12.169338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:12.281607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 952/1891] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:12.293714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:12.378444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:12.382663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:12.450697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f04c1-74b2-4fc1-89db-338f013a322c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:12.455041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:12.455670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.455953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.456227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.472558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:12.474408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:12.489148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 940/1892] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:12.739842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:12.802323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:12.807010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:12.816865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9cec2-7d8f-4694-ae61-71b99ca08129 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:12.818038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbc695-a89d-4e1e-b770-41384f7fe7ed None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:12.822207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:12.822903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.823217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.823472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.825604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:12.826493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:12.827499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:12.827912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:12.828514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:12.904548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:12.905963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf6b1e-1216-4b99-851b-e7ef8001c5da service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:12.908225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:12.910942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7814d4c-2dc7-4014-875c-5ef7d4573a8c service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:12.918460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 941/1893] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:12.930346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 953/1894] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:14.590878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:14.591591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:14.591591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:14.593025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 942/1895] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:17:14.650307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:14.651319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:14.651856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:14.670776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:14.685032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:14.686293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:14.771061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 954/1896] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:17:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:14.813351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:14.815777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:14.838097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f76f-0053-4411-a6d5-c95c0669763d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:14.848512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:14.849310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:14.849629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:14.849973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:14.862312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:14.866566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:14.884890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 943/1897] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:15.147377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:19.173092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:19.853880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:22.104776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:23.045741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:27.308992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:28.071099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:29.199582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:29.385289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:29.395001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24776053-ebbf-4f32-810f-08615ab40689 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:29.406800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:29.406800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:29.406800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:29.406800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:29.407196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:29.463510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:29.465000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91585c4-a857-479c-818a-1d0bd4a8e861 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:29.474028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 955/1898] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:32.419558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:32.824334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:34.178184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:34.854421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:34.956391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7eb02-7c8d-4d1f-93ec-4b97014f57ab service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:34.961608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:34.962532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:34.962844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:34.963284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] PATH_INFO: `/v3/users/0b5ed82057184a0ea2c2453a346a4900` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:34.964622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5ed82057184a0ea2c2453a346a4900)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:34.966752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:35.249542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5ed82057184a0ea2c2453a346a4900'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:35.250525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5ed82057184a0ea2c2453a346a4900'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:35.269340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5ed82057184a0ea2c2453a346a4900'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:35.440215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5ed82057184a0ea2c2453a346a4900'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:35.450510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5ed82057184a0ea2c2453a346a4900'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:35.467570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 944/1899] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:34 2022] DELETE /identity/v3/users/0b5ed82057184a0ea2c2453a346a4900 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:35.477325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:35.477851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:35.478233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:35.499708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:35.500987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:35.572062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 956/1900] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:35.585470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:35.587419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:35.685239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad782c-7630-4554-b004-dc5101db55b6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:35.693148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:35.694539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:35.694849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:35.695198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:35.706110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:35.707786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:35.719701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 945/1901] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:35.904175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:36.369610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:36.370907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:36.385338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bebf5-bbdf-419d-995a-837ba70e3643 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:36.390899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:36.391565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:36.391850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:36.392128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] PATH_INFO: `/v3/projects/74820d32d28e4657ae674b3c1ab7a32b` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:36.393359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74820d32d28e4657ae674b3c1ab7a32b)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:36.395345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:36.694741 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74820d32d28e4657ae674b3c1ab7a32b', 'request_id': 'req-ee006cc9-5ce4-4b53-83bf-93ac40453f98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:36.700251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74820d32d28e4657ae674b3c1ab7a32b', 'request_id': 'req-ee006cc9-5ce4-4b53-83bf-93ac40453f98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:36.928826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74820d32d28e4657ae674b3c1ab7a32b', 'request_id': 'req-ee006cc9-5ce4-4b53-83bf-93ac40453f98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:36.943654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] The token cache is being invalidate because project 74820d32d28e4657ae674b3c1ab7a32b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:36.943963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:36.944846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:36.949770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 957/1902] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:36 2022] DELETE /identity/v3/projects/74820d32d28e4657ae674b3c1ab7a32b => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:36.951358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:36.963192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3df5ee-c1d7-4cb7-bad0-c017256c66c7 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:37.035501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:37.036525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:37.036815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:37.037585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] PATH_INFO: `/v3/users/2ef43401f0124d4dab2858f735c37a47` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:37.038905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef43401f0124d4dab2858f735c37a47)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:37.040921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:37.253431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:37.534176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef43401f0124d4dab2858f735c37a47'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:37.534902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef43401f0124d4dab2858f735c37a47'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:37.547983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef43401f0124d4dab2858f735c37a47'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:37.789309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef43401f0124d4dab2858f735c37a47'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:37.794505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef43401f0124d4dab2858f735c37a47'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:37.800696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 946/1903] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:36 2022] DELETE /identity/v3/users/2ef43401f0124d4dab2858f735c37a47 => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:38.609080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:38.610565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:38.624030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee006cc9-5ce4-4b53-83bf-93ac40453f98 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:38.694154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:38.695704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:38.695704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:38.695704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] PATH_INFO: `/v3/projects/e335343f5ba54c57b093091b3d3a2e49` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:38.696391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e335343f5ba54c57b093091b3d3a2e49)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:38.698352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:38.831532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:38.833197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:38.846607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d51620-1a46-4e8c-83be-107c930b381c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:38.856095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:38.856095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:38.856095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:38.856095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] PATH_INFO: `/v3/users/a44102a2fcce4c3999860e15dfb6a6b5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:38.858821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44102a2fcce4c3999860e15dfb6a6b5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:38.860185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:38.870908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e335343f5ba54c57b093091b3d3a2e49', 'request_id': 'req-027ec4c5-0f64-4b21-8414-e471dbe2b465'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:38.877393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e335343f5ba54c57b093091b3d3a2e49', 'request_id': 'req-027ec4c5-0f64-4b21-8414-e471dbe2b465'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:38.996389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e335343f5ba54c57b093091b3d3a2e49', 'request_id': 'req-027ec4c5-0f64-4b21-8414-e471dbe2b465'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.010272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] The token cache is being invalidate because project e335343f5ba54c57b093091b3d3a2e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:39.010931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.011664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-027ec4c5-0f64-4b21-8414-e471dbe2b465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.013214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 958/1904] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:38 2022] DELETE /identity/v3/projects/e335343f5ba54c57b093091b3d3a2e49 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:39.018342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.018643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.018955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.041843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:39.043373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.124770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 959/1905] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:39.125557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.126918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.138494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a62927-fcd1-4cc9-876a-269f0c7e755e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.143040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.143669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.143954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.144262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] PATH_INFO: `/v3/users/5f2e7d35221e474ba4319ccfeb7704d1` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.145397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2e7d35221e474ba4319ccfeb7704d1)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.147974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.180477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44102a2fcce4c3999860e15dfb6a6b5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.181156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44102a2fcce4c3999860e15dfb6a6b5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.196295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44102a2fcce4c3999860e15dfb6a6b5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.325677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44102a2fcce4c3999860e15dfb6a6b5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.337632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-907eb0c5-7531-498c-b379-ed27c9da694c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44102a2fcce4c3999860e15dfb6a6b5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.345995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 947/1906] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:38 2022] DELETE /identity/v3/users/a44102a2fcce4c3999860e15dfb6a6b5 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:39.347080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a364-c7b3-43a0-a489-20723f569f18 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.347363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a364-c7b3-43a0-a489-20723f569f18 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.347698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079a364-c7b3-43a0-a489-20723f569f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.370329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3079a364-c7b3-43a0-a489-20723f569f18 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:39.371593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3079a364-c7b3-43a0-a489-20723f569f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.438223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 948/1907] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:39.439697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.439983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.440260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.466140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:39.468416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.477288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 949/1908] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:39.478153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.488298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f539b7-8242-406e-b68f-8ac457bdf38d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.493698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.494323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.494608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.494910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.495605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:39.495908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.497555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.518557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 950/1909] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:39.518557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.518557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.541358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2e7d35221e474ba4319ccfeb7704d1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.542059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2e7d35221e474ba4319ccfeb7704d1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.554792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2e7d35221e474ba4319ccfeb7704d1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.592993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f08f4b-8570-40cf-91c4-956b51f1cead admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.597086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.597785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.598187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.598393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.600805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.613687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.615171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.626012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 951/1910] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:39.626859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.628204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.640748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77b12f-f3d7-4ce9-85b0-a0e839289d19 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.645396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2e7d35221e474ba4319ccfeb7704d1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.646077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.646238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.646666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.647563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.648555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.651223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.655874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d3a7e261-c79c-4a38-9bfb-84c3ce5f2f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2e7d35221e474ba4319ccfeb7704d1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:39.664722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 960/1911] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:39 2022] DELETE /identity/v3/users/5f2e7d35221e474ba4319ccfeb7704d1 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:39.669423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.670149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.670497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.690104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:39.691482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.756848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 961/1912] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:39.767505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.770030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:39.857188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8e7af-c716-4376-9ad3-0beeea8bed32 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.857291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 952/1913] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:39.858985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:39.910658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:39.928402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.929627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.930068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.930670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.932969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf80a7-ee40-4014-a91f-b96dd792e993 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:39.938635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:39.939264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:39.940212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:39.940212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:39.940522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.942472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.945338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:39.947030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:39.963185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 962/1914] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:40.122852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 953/1915] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:39 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:40.124430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.136038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a865ec4-7a6b-4bf5-b1f6-d9ed1cbca306 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.142196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.142836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.143122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.143402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.144075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.145668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.160717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 963/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:40.165142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.192288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d7d84-328b-42bc-a3c5-b39961b44c56 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.198318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.199294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.199583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.199859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.200520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.202143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.214151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 954/1917] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:40.218317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.241059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d354d7-67a0-4e13-a583-1d106e889421 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.245922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.246905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.247379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.247892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] PATH_INFO: `/v3/projects/8d70052c67404d3faf82f6edb6d692ce/users/83622c48aaa84cdd9930b674df1791e6/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.267586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d70052c67404d3faf82f6edb6d692ce, user_id=83622c48aaa84cdd9930b674df1791e6, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.270405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d90d1-abf8-4dc8-892a-956310dd020c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.309718 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.311554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:40.325362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b0fe8-26fa-4cd6-bc61-9040e675f404 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.330517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.331268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.344858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.344858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] PATH_INFO: `/v3/projects/c348892002ae4e3d81f8737ad8ce6aa2` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.344858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c348892002ae4e3d81f8737ad8ce6aa2)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.344858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.459933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 964/1918] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:40 2022] PUT /identity/v3/projects/8d70052c67404d3faf82f6edb6d692ce/users/83622c48aaa84cdd9930b674df1791e6/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:40.471822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.472119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.472404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.502407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] MFA Rules not processed for user `83622c48aaa84cdd9930b674df1791e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:40.503702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:40.546366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c348892002ae4e3d81f8737ad8ce6aa2', 'request_id': 'req-09ef3d0f-0098-4888-bde4-678663b818a9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.610846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 965/1919] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:40.611719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.616630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:40.663680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c348892002ae4e3d81f8737ad8ce6aa2', 'request_id': 'req-09ef3d0f-0098-4888-bde4-678663b818a9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.679609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c348892002ae4e3d81f8737ad8ce6aa2', 'request_id': 'req-09ef3d0f-0098-4888-bde4-678663b818a9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.690420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967b30f-790c-41d7-b93d-1910e61325e9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.711748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] The token cache is being invalidate because project c348892002ae4e3d81f8737ad8ce6aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:40.712308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.713094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.719736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 955/1920] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:40 2022] DELETE /identity/v3/projects/c348892002ae4e3d81f8737ad8ce6aa2 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:40.720758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:40.722560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:40.757494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.758147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.758516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.758840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] PATH_INFO: `/v3/projects/e24c359a18a54b3dbd3ccc2b2d79c20f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.760086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24c359a18a54b3dbd3ccc2b2d79c20f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.762020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.793047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3d0f-0098-4888-bde4-678663b818a9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:40.800151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:40.800860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:40.801185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:40.801500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:40.802178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:40.805156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:40.967175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24c359a18a54b3dbd3ccc2b2d79c20f', 'request_id': 'req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:40.970865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24c359a18a54b3dbd3ccc2b2d79c20f', 'request_id': 'req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.140693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 956/1921] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:40 2022] POST /identity/v3/projects => generated 385 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:41.141428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.143247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:41.196234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24c359a18a54b3dbd3ccc2b2d79c20f', 'request_id': 'req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.209484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] The token cache is being invalidate because project e24c359a18a54b3dbd3ccc2b2d79c20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:41.210020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.210913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.212707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 966/1922] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:40 2022] DELETE /identity/v3/projects/e24c359a18a54b3dbd3ccc2b2d79c20f => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:41.213487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.215009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:41.223715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fd363-542c-46f6-b74f-fb5e59ebadca admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:41.229193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:41.229854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:41.230133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:41.230407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] PATH_INFO: `/v3/users/43b0994a9fcf4062acbc4435552e71d0` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:41.231651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b0994a9fcf4062acbc4435552e71d0)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:41.233629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:41.285727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213b19-7ac9-4689-a740-6f06cdc7bfe1 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:41.290579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:41.291424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:41.291711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:41.291992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:41.292886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:41.294895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:41.451782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 967/1923] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:41 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:41.451782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.451782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:41.464423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac40ba9-c5ee-4d59-9b6a-b47698c14361 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:41.469699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:41.470343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:41.470648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:41.470947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] PATH_INFO: `/v3/users/f3fec9e5844d471f8060ee61934de0b0` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:41.472103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fec9e5844d471f8060ee61934de0b0)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:41.474195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:41.552568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b0994a9fcf4062acbc4435552e71d0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.553984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b0994a9fcf4062acbc4435552e71d0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.565903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b0994a9fcf4062acbc4435552e71d0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.687439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b0994a9fcf4062acbc4435552e71d0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.695545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b0994a9fcf4062acbc4435552e71d0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.701826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 957/1924] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:41 2022] DELETE /identity/v3/users/43b0994a9fcf4062acbc4435552e71d0 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:41.702710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.704137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:41.773671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf44157-8483-452c-a05f-749597de8b63 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:41.779171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:41.779815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:41.780103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:41.780445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:41.781143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:41.782827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:41.795720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 958/1925] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:41.799435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.805705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fec9e5844d471f8060ee61934de0b0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.806484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fec9e5844d471f8060ee61934de0b0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.812626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549d9cc-3bb7-464a-a590-40f58351109f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:41.817990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:41.818732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:41.819022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:41.820552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] PATH_INFO: `/v3/projects/ad240654f1134efe941f54ff577768a5/users/de7d931aa3f04fbaac8b2e386bf2f1e5/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:41.821011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fec9e5844d471f8060ee61934de0b0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.828382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:41.837647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad240654f1134efe941f54ff577768a5, user_id=de7d931aa3f04fbaac8b2e386bf2f1e5, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:41.840508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:41.967427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fec9e5844d471f8060ee61934de0b0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.978215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fec9e5844d471f8060ee61934de0b0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:41.981802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 959/1926] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:41 2022] PUT /identity/v3/projects/ad240654f1134efe941f54ff577768a5/users/de7d931aa3f04fbaac8b2e386bf2f1e5/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:41.982820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:41.984517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.004029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 968/1927] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:41 2022] DELETE /identity/v3/users/f3fec9e5844d471f8060ee61934de0b0 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:42.058857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e864fc-b7f0-4e57-9be8-a3285fc33c79 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:42.138572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:42.139230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:42.139828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:42.139828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:42.140457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:42.142075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93338a48-9c81-4983-8ae4-72b45f90adce admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:42.152933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 960/1928] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:42.157098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:42.167449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2869e-454b-4c73-91b2-b270e7d8b9db admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:42.172343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:42.172994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:42.173277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:42.173552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] PATH_INFO: `/v3/projects/ad240654f1134efe941f54ff577768a5/users/de7d931aa3f04fbaac8b2e386bf2f1e5/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:42.175511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad240654f1134efe941f54ff577768a5, user_id=de7d931aa3f04fbaac8b2e386bf2f1e5, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:42.178259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:42.188458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 969/1929] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:42 2022] PUT /identity/v3/projects/ad240654f1134efe941f54ff577768a5/users/de7d931aa3f04fbaac8b2e386bf2f1e5/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:42.200976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:42.201273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:42.201556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:42.231305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] MFA Rules not processed for user `de7d931aa3f04fbaac8b2e386bf2f1e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:42.232543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.303539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 961/1930] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:42.315660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:42.317930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.377875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700ac13-033f-40bc-91bc-cd10c1c03d65 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:42.382260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:42.382899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:42.383187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:42.383464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:42.385194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.445032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:42.446587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:42.455771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 970/1931] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:42.494445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:42.496080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.710543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a03890-31cf-4d4f-9cbb-14a3346a410e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:42.710543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:42.710543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:42.710543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:42.712987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] PATH_INFO: `/v3/projects/1d1fff03fa7240b89e06f388faf280e2` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:42.712987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1fff03fa7240b89e06f388faf280e2)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:42.712987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:42.969883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:42.973626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:42.998449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1fff03fa7240b89e06f388faf280e2', 'request_id': 'req-b56ed87f-4908-4d2c-b417-38f814c5163f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.027967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7040e1-b6a4-48ad-a56a-0326d0cfedab service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:43.123594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:43.123884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.124174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.124484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] PATH_INFO: `/v3/projects/21d575f745694550acd890029cf13436` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.125696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d575f745694550acd890029cf13436)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:43.128041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:43.156950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1fff03fa7240b89e06f388faf280e2', 'request_id': 'req-b56ed87f-4908-4d2c-b417-38f814c5163f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.168148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1fff03fa7240b89e06f388faf280e2', 'request_id': 'req-b56ed87f-4908-4d2c-b417-38f814c5163f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.183193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] The token cache is being invalidate because project 1d1fff03fa7240b89e06f388faf280e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:43.183575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.184463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.186045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 962/1932] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:42 2022] DELETE /identity/v3/projects/1d1fff03fa7240b89e06f388faf280e2 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:43.190276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:43.191729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:43.206883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ed87f-4908-4d2c-b417-38f814c5163f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:43.212092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:43.212709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.212995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.213275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] PATH_INFO: `/v3/users/d4c4fda878bc4dd2a5dd43f2fdb63569` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.214423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c4fda878bc4dd2a5dd43f2fdb63569)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:43.216439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:43.299821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d575f745694550acd890029cf13436', 'request_id': 'req-6aca2590-7e98-4367-b6f4-d5018ec1cad0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.310809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d575f745694550acd890029cf13436', 'request_id': 'req-6aca2590-7e98-4367-b6f4-d5018ec1cad0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.441903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d575f745694550acd890029cf13436', 'request_id': 'req-6aca2590-7e98-4367-b6f4-d5018ec1cad0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.456272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] The token cache is being invalidate because project 21d575f745694550acd890029cf13436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:43.456571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.457303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6aca2590-7e98-4367-b6f4-d5018ec1cad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.463378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 971/1933] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:42 2022] DELETE /identity/v3/projects/21d575f745694550acd890029cf13436 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:43.464967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f967-f67f-46ea-8ef9-f1f71b6f9d1a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.465262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f967-f67f-46ea-8ef9-f1f71b6f9d1a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.466150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f967-f67f-46ea-8ef9-f1f71b6f9d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.498250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3ce3f967-f67f-46ea-8ef9-f1f71b6f9d1a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:43.499584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3ce3f967-f67f-46ea-8ef9-f1f71b6f9d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:43.576368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c4fda878bc4dd2a5dd43f2fdb63569'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.578274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c4fda878bc4dd2a5dd43f2fdb63569'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.602647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 972/1934] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:43.603685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c4fda878bc4dd2a5dd43f2fdb63569'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.612061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.612437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.612776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.636286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:43.642937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:43.647748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 973/1935] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:43.651800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:43.668738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0b0e7-c18d-4442-8e06-4d00a4e965a0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:43.675893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:43.676605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.678314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.678314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.678314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:43.678314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:43.680197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:43.694205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 974/1936] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:43.696732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c4fda878bc4dd2a5dd43f2fdb63569'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.701697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:43.708255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c4fda878bc4dd2a5dd43f2fdb63569'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:43.718133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 963/1937] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:43 2022] DELETE /identity/v3/users/d4c4fda878bc4dd2a5dd43f2fdb63569 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:43.721914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d8d33-68b1-46ae-87f7-45757a63674d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:43.816745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:43.817534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:43.817875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:43.818506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:43.820168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:43.822906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:43.885404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:43.932732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 975/1938] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:43 2022] POST /identity/v3/projects => generated 369 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:43.938128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:43.949890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500996b2-1d37-45a3-b880-65d345e0ab3d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.030349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.031024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.031315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.031592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.032356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.034446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.239042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 964/1939] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:43 2022] POST /identity/v3/users => generated 285 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:44.239981 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.242296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:44.245124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.258143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d47c-7ab9-4e36-89cf-ad388af81b9b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.263482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a761ec-05a2-4938-ae10-c63ff4174a37 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.271154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.273211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.273510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.273777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.274451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.275124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.278286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.278506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.279334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.280044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] PATH_INFO: `/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.285122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41e8c1ff5514a2699fc2cfda3d89a19)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.291207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.293723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 965/1940] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:44.299780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.312815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201cfbb-558f-477a-865d-716ece9023f9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.320054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.320815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.321098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.321378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.323946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.324261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.340027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 966/1941] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:44.345695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.362669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6c1a3-3d11-4c98-81bc-83dffce6c980 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.370907 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.371908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.372357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.372700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] PATH_INFO: `/v3/projects/81d9b877ad9442c3b9f8718ef4b60ff9/users/320d8d4c10c4427d86c36cce38a88bd2/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.398354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d9b877ad9442c3b9f8718ef4b60ff9, user_id=320d8d4c10c4427d86c36cce38a88bd2, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.401285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cb57c-ca58-4862-af44-ed622d1d5821 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.546142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 967/1942] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:44 2022] PUT /identity/v3/projects/81d9b877ad9442c3b9f8718ef4b60ff9/users/320d8d4c10c4427d86c36cce38a88bd2/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:44.556950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.557467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.557892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.567903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41e8c1ff5514a2699fc2cfda3d89a19', 'request_id': 'req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:44.577161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41e8c1ff5514a2699fc2cfda3d89a19', 'request_id': 'req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:44.595225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] MFA Rules not processed for user `320d8d4c10c4427d86c36cce38a88bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:44.596811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:44.650846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41e8c1ff5514a2699fc2cfda3d89a19', 'request_id': 'req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:44.672478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] The token cache is being invalidate because project f41e8c1ff5514a2699fc2cfda3d89a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:44.672478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:44.672478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3b88bfa7-c0ab-4a6e-bccc-fa9dfbd84ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:44.672478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 976/1943] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:44 2022] DELETE /identity/v3/projects/f41e8c1ff5514a2699fc2cfda3d89a19 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:44.672774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ffed-7a85-4105-a548-9de0a3f0654b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.672994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ffed-7a85-4105-a548-9de0a3f0654b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.673374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944ffed-7a85-4105-a548-9de0a3f0654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.675635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 968/1944] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:44.680085 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.695720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e944ffed-7a85-4105-a548-9de0a3f0654b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:44.697162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e944ffed-7a85-4105-a548-9de0a3f0654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:44.749834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb4303-cbf8-47e0-8209-3038155564c0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.757459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.758083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.758370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.758800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.759468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.761400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.777031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 977/1945] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:44.788085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.788593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.799349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.811088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:44.812618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:44.823630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 978/1946] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:44.827963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.838166 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb3ac9-0e3f-4703-a1fd-87a34169a822 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.842905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.843573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.843856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.844127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.844852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:44.845159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.846817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.859521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 979/1947] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:17:44.864128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.874491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f18a1-33b5-42b1-8b86-b99ae01cf5f4 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.879973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.880592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.881425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.881722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.882394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:44.884475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:44.902276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 969/1948] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:44.904049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:44.905685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:44.989033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695e030-ad40-4c73-98a7-5a23b5edc82c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:44.998105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:44.998105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:44.998105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:44.998105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:44.998532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.000603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 980/1949] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/projects => generated 391 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:17:45.001139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.001523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.067062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa6e14-c63e-4fa9-8dbb-17ebbbef5538 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.073256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.073931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.074281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.074566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.075251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.077406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.182282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 970/1950] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:44 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:45.185468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.196958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796644e4-84f4-4425-8b48-1f2a9f59d07b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.202250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.202921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.203207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.203487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.204674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.205941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.225005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 971/1951] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:45.225005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.230752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 981/1952] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:45 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:45.231747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.240046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c7b25-15e4-417e-8e03-6a17794222e1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.254610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530928f-d664-4771-9707-1af2c6c47b0a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.259532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.260333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.260696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.260983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] PATH_INFO: `/v3/projects/45b55353ab5040c3b0adfa5288fcafec/users/0f60559bb5404f4ebc643c7b5a65459f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.264215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.264215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.264215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.264215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.264688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.265417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.282865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 982/1953] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:45.294931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.295086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45b55353ab5040c3b0adfa5288fcafec, user_id=0f60559bb5404f4ebc643c7b5a65459f, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.306726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278065-8200-4efa-aeca-c23adb05d364 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.306926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.312836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.313814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.314260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.314762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.315431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.317076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.332924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 983/1954] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:45.337075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.346997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0a33d-e448-4d55-94e1-859bf334df20 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.352462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.353121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.353405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.353794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] PATH_INFO: `/v3/projects/0ea1f4e8f8574ff58662f84f62d36464/users/076b5ffacff049d7a2694260ffd08d59/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.363838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:45.372735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea1f4e8f8574ff58662f84f62d36464, user_id=076b5ffacff049d7a2694260ffd08d59, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.375436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427cb0-8432-452d-8eeb-1b56080db79c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.461077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 972/1955] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:45 2022] PUT /identity/v3/projects/45b55353ab5040c3b0adfa5288fcafec/users/0f60559bb5404f4ebc643c7b5a65459f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:45.466607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.467634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 984/1956] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:45 2022] PUT /identity/v3/projects/0ea1f4e8f8574ff58662f84f62d36464/users/076b5ffacff049d7a2694260ffd08d59/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:45.479477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.479762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.480108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.523052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] MFA Rules not processed for user `076b5ffacff049d7a2694260ffd08d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:45.529652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:45.563652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5ef3-f2f0-4e9f-8777-af63c61f9db4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.641739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 985/1957] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:45.643037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.655312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.656542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.657268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.657712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.658672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.660315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.681023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 973/1958] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:45.682086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.694492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e9056-cf17-4563-a5a3-8b18cca2bcdd admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.701165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.702824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.703317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.703748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] PATH_INFO: `/v3/projects/45b55353ab5040c3b0adfa5288fcafec/users/0f60559bb5404f4ebc643c7b5a65459f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.706552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45b55353ab5040c3b0adfa5288fcafec, user_id=0f60559bb5404f4ebc643c7b5a65459f, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.711439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fbc49-823d-48b1-a0aa-f68c970664d8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.736813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4bb56-cd00-4f09-8918-ab71a2e79716 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:45.739142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 974/1959] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:45 2022] PUT /identity/v3/projects/45b55353ab5040c3b0adfa5288fcafec/users/0f60559bb5404f4ebc643c7b5a65459f/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:45.754006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.754301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.754580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.792287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] MFA Rules not processed for user `0f60559bb5404f4ebc643c7b5a65459f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:45.793965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:45.816716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:45.818097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:45.818760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:45.819327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:45.820753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:45.835993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:45.910695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 975/1960] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:45.937826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:45.939381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:46.007828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 986/1961] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:45 2022] POST /identity/v3/projects => generated 391 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:46.013452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.024925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88984332-e823-48c4-b53d-ddd0dd263b42 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.087822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.089893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.090175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.090448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.104776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c936e-5cf9-4569-87d7-44f997ef0479 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.111040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.112007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.112489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.112977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.114041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.116826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.181084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.184106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.195664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 976/1962] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:46.261044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 987/1963] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:46 2022] POST /identity/v3/users => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:46.265132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.276305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cdfc4-f16a-450d-b823-74de0a4bfcf1 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.282822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.283609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.283860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.284133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.284790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.286428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.299269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 977/1964] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:46.304171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.316387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0db7d-bbf0-4831-8859-98378a382284 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.328825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.329890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.330385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.331051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] PATH_INFO: `/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed/users/383672bf334f4e65afbef02d227d801a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.361523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5bf311dc4942e6bbe8a7aa3ea200ed, user_id=383672bf334f4e65afbef02d227d801a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.364302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.407929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.422009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a4ac0-a28d-4002-bcd5-15277917e258 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.427168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.433575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.433575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.433575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.433575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:46.461007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 988/1965] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:46 2022] PUT /identity/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed/users/383672bf334f4e65afbef02d227d801a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:46.469490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.535893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.538845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.549110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b94d5-4591-4060-b7a2-50d4674b14a9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.552145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 978/1966] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:46.556779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.557909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.558563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.559232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.560507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.563746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460e8d3-2dd5-410c-9349-5b3a90798584 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.575135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 989/1967] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:17:46.579960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.592025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73035862-e44e-4844-bc6b-d37d20139e08 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.597090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.597707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.597978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.598256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] PATH_INFO: `/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed/users/383672bf334f4e65afbef02d227d801a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.600431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5bf311dc4942e6bbe8a7aa3ea200ed, user_id=383672bf334f4e65afbef02d227d801a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.603882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.614319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 979/1968] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:46 2022] PUT /identity/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed/users/383672bf334f4e65afbef02d227d801a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:46.629607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.629936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.630221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.664524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] MFA Rules not processed for user `383672bf334f4e65afbef02d227d801a`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:46.665850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:46.773358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 990/1969] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:46.796828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:46.799221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:46.833290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:46.878335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572ef1b-39b7-41db-9a2e-32fcf35e2e43 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:46.883716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:46.886313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:46.886313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:46.886313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:46.908723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:46.910424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:46.929750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 980/1970] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:47.055829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:47.069033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f677-6337-4311-80c5-ea6ab0867689 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:47.074200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:47.074856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:47.075141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:47.075415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:47.142009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:47.144007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:47.153642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 991/1971] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:48.148326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:49.556323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:50.803538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:50.815976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6f055-0f80-4f3f-9ecc-5c9f5a168c62 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:50.822253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:50.823360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:50.823862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:50.824329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:50.827173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:50.904217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:50.905818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6871db-278a-4a12-b168-0b80e30bf0b4 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:50.925538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 981/1972] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:51.228995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:51.652868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:51.652868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:51.889232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b6f23-f3a0-4776-81bd-8f2eb2ef24c4 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:51.897464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:51.905280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:51.905574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:51.908221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] PATH_INFO: `/v3/users/49331867358f4bf08e4ea7b25c49ad2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:51.911064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49331867358f4bf08e4ea7b25c49ad2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:51.915533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:52.163355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49331867358f4bf08e4ea7b25c49ad2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:52.164027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49331867358f4bf08e4ea7b25c49ad2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:52.180966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49331867358f4bf08e4ea7b25c49ad2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:52.264510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:52.310838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49331867358f4bf08e4ea7b25c49ad2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:52.328567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49331867358f4bf08e4ea7b25c49ad2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:52.343876 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 992/1973] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:51 2022] DELETE /identity/v3/users/49331867358f4bf08e4ea7b25c49ad2f => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:52.352801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:52.352801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:52.352801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:52.387307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:52.389259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:52.487415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 982/1974] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:52.503397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:52.505751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:52.578380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4148416-3d01-46dc-ae2c-6de1e0922327 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:52.582252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:52.582899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:52.583181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:52.583455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:52.595475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:52.596940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:52.608361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 993/1975] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:52.865124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:53.196363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:53.197739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:53.211730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37321-ef24-4433-947c-33c107d62496 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:53.217292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:53.217929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:53.218471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:53.218471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] PATH_INFO: `/v3/users/8edb7f9dc184486e8fd296fb37898311` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:53.226945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8edb7f9dc184486e8fd296fb37898311)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:53.229824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:53.276176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:53.278154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:53.299864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f6cb-09f9-4122-9d02-508d31e92028 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:53.319801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:53.320722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:53.321014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:53.321297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] PATH_INFO: `/v3/projects/5b8dffc1266c42ad9369571561a715fd` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:53.322703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8dffc1266c42ad9369571561a715fd)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:53.325023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:53.603540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8edb7f9dc184486e8fd296fb37898311'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.604736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8edb7f9dc184486e8fd296fb37898311'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.614606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8dffc1266c42ad9369571561a715fd', 'request_id': 'req-2e3ebb16-aeef-442f-b854-68198b9d5831'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.627165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8edb7f9dc184486e8fd296fb37898311'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.627539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8dffc1266c42ad9369571561a715fd', 'request_id': 'req-2e3ebb16-aeef-442f-b854-68198b9d5831'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.784604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8edb7f9dc184486e8fd296fb37898311'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.797916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b6b04248-5c5e-4c88-954a-156a6315fcf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8edb7f9dc184486e8fd296fb37898311'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.806421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 983/1976] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:53 2022] DELETE /identity/v3/users/8edb7f9dc184486e8fd296fb37898311 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:53.811612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbcf7a-5b80-4c77-8704-29c1470da9bf None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:53.811909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbcf7a-5b80-4c77-8704-29c1470da9bf None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:53.812161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbcf7a-5b80-4c77-8704-29c1470da9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:53.837073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9bfbcf7a-5b80-4c77-8704-29c1470da9bf None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:53.838498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9bfbcf7a-5b80-4c77-8704-29c1470da9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:53.903584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8dffc1266c42ad9369571561a715fd', 'request_id': 'req-2e3ebb16-aeef-442f-b854-68198b9d5831'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.924041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 984/1977] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:53.925714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] The token cache is being invalidate because project 5b8dffc1266c42ad9369571561a715fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:53.926041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.926757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:53.929349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 994/1978] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:53 2022] DELETE /identity/v3/projects/5b8dffc1266c42ad9369571561a715fd => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:53.934167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5c20-4d58-4231-b1ab-924f96c04a26 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:53.934558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5c20-4d58-4231-b1ab-924f96c04a26 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:53.934868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5c20-4d58-4231-b1ab-924f96c04a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:53.937989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:53.939898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:53.960554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-933f5c20-4d58-4231-b1ab-924f96c04a26 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:53.961516 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-933f5c20-4d58-4231-b1ab-924f96c04a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:53.970608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 985/1979] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:53.980110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:53.980391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:53.980665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.013534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:54.019485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:54.030523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 986/1980] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:17:54.033261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.038838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ebb16-aeef-442f-b854-68198b9d5831 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.050097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f6b90-8b20-4240-93a5-346fac077c30 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.056949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.057765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.058418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.058418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.059104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:54.059402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.061451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.090273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 987/1981] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:17:54.091287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.102581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c9ff-d364-4546-bb5d-2085ea3b00f3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.108281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.109069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.109457 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.110679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.110679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.114415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.131469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.132659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.133232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.133798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.137399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:54.157682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.159198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.182232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 995/1982] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:17:54.384956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 988/1983] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:54 2022] POST /identity/v3/projects => generated 373 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:54.390883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.404333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3140e01-643b-43bb-895c-0a11ed70038f service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.493172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.493778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.494065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.494530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.495195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.497118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.687822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 996/1984] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:17:54 2022] POST /identity/v3/users => generated 287 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:54.692894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.703900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652bb52-5ca9-4c90-8281-e6dbf5e373a7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.710018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.710781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.710909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.711122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.711778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.713397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.723960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 989/1985] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:54.739071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.750646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2659-c960-4402-9845-1bad37794ec3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.756617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.757674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.757674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.758032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.758910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.760556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.774663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 997/1986] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:17:54.777621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.777751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:54.779576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:54.789175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eb8b0-a195-4618-9636-d2f0ec769abd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.794291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.795202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.795359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.795639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] PATH_INFO: `/v3/projects/9930af8bcf7f4bdbb00ee6b3a9ca22db/users/d191f7381b2e48ab979fcdb672b9e33f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.798489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c5adf-e6d4-44d3-8b04-669b7d20d688 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:54.804540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:54.805353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:54.805629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:54.806189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] PATH_INFO: `/v3/users/076b5ffacff049d7a2694260ffd08d59` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:54.807366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=076b5ffacff049d7a2694260ffd08d59)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.809375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:54.814931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9930af8bcf7f4bdbb00ee6b3a9ca22db, user_id=d191f7381b2e48ab979fcdb672b9e33f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:54.817855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:55.132389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 998/1987] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:17:54 2022] PUT /identity/v3/projects/9930af8bcf7f4bdbb00ee6b3a9ca22db/users/d191f7381b2e48ab979fcdb672b9e33f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:55.132389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:55.132389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.220623 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b88b34-f53a-4237-b99e-0b550be9e42f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:55.228899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:55.229950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.230445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.230769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] PATH_INFO: `/v3/projects/b1b11d5ed90b49e29785f7df4b2ddc48` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.231952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b11d5ed90b49e29785f7df4b2ddc48)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:55.234046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:55.294375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076b5ffacff049d7a2694260ffd08d59'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.294375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076b5ffacff049d7a2694260ffd08d59'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.310785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076b5ffacff049d7a2694260ffd08d59'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.385782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:55.387493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076b5ffacff049d7a2694260ffd08d59'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.399582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2a95078d-9ee4-429d-a939-cb2df36a7d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076b5ffacff049d7a2694260ffd08d59'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.407279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 990/1988] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:54 2022] DELETE /identity/v3/users/076b5ffacff049d7a2694260ffd08d59 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:55.409095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512958c-54e3-478b-96af-2b74824c57ef None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.409386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512958c-54e3-478b-96af-2b74824c57ef None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.409724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512958c-54e3-478b-96af-2b74824c57ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.423436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b11d5ed90b49e29785f7df4b2ddc48', 'request_id': 'req-249c6278-8e76-4842-a163-ac9b365bbd9d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.437429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b11d5ed90b49e29785f7df4b2ddc48', 'request_id': 'req-249c6278-8e76-4842-a163-ac9b365bbd9d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.451716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1512958c-54e3-478b-96af-2b74824c57ef None None] MFA Rules not processed for user `d191f7381b2e48ab979fcdb672b9e33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:55.453073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1512958c-54e3-478b-96af-2b74824c57ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.543767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 991/1989] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:55.544764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.545078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.545315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.566488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:55.567746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.628794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b11d5ed90b49e29785f7df4b2ddc48', 'request_id': 'req-249c6278-8e76-4842-a163-ac9b365bbd9d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.667034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 992/1990] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:55.672636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] The token cache is being invalidate because project b1b11d5ed90b49e29785f7df4b2ddc48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:55.673275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.674443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:55.681325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 999/1991] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:55 2022] DELETE /identity/v3/projects/b1b11d5ed90b49e29785f7df4b2ddc48 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:55.682675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:55.684864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.688291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:55.689792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.786577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c6278-8e76-4842-a163-ac9b365bbd9d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:55.789697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552700e-04ee-4fd2-8415-d14f576efdc4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:55.796334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:55.797266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.797748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.797913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:55.798415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] PATH_INFO: `/v3/users/d02467ec4b9b46ae91d4467073a79768` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.798581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.798957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.799258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.800522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02467ec4b9b46ae91d4467073a79768)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:55.801166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.805071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:55.822055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:55.824669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:55.846017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 993/1992] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:55.847886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:55.850966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.930775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408e060-da59-418b-bbd8-005f4581fad6 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:55.934845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:55.935479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:55.935825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:55.936665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:55.939820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.955711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:55.957277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:55.973525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 994/1993] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:55.975562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:55.977087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:55.995088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbb22-9b96-415c-a5d1-5c90214d7aa9 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:56.001341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:56.002153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:56.002610 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:56.002982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] PATH_INFO: `/v3/users/320d8d4c10c4427d86c36cce38a88bd2` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:56.004584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=320d8d4c10c4427d86c36cce38a88bd2)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:56.008117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:56.135174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02467ec4b9b46ae91d4467073a79768'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.136510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02467ec4b9b46ae91d4467073a79768'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.156000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02467ec4b9b46ae91d4467073a79768'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.286094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02467ec4b9b46ae91d4467073a79768'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.301912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02467ec4b9b46ae91d4467073a79768'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.310360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1000/1994] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:55 2022] DELETE /identity/v3/users/d02467ec4b9b46ae91d4467073a79768 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:56.443047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320d8d4c10c4427d86c36cce38a88bd2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.455409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320d8d4c10c4427d86c36cce38a88bd2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:56.462558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320d8d4c10c4427d86c36cce38a88bd2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.161323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:57.163634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:57.216175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320d8d4c10c4427d86c36cce38a88bd2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.230599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3025d292-25c0-44b7-9ded-fcf621afce6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320d8d4c10c4427d86c36cce38a88bd2'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.243742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 995/1995] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:55 2022] DELETE /identity/v3/users/320d8d4c10c4427d86c36cce38a88bd2 => generated 0 bytes in 1270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:57.249953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:57.251025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:57.251845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:57.281925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:57.283828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:57.298853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ffbef-5fa5-40f5-b91d-dcb5bc34c717 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:57.380139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 996/1996] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:57.395382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:57.399288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:57.427478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:57.428763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:57.429002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:57.429281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] PATH_INFO: `/v3/projects/0ea1f4e8f8574ff58662f84f62d36464` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:57.430692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea1f4e8f8574ff58662f84f62d36464)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:57.432647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:57.509719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd917584-8c96-4191-a65a-80c7f25e7145 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:57.515286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:57.516579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:57.516579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:57.516923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:57.529954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:57.531746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:57.550034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 997/1997] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:17:57.702048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea1f4e8f8574ff58662f84f62d36464', 'request_id': 'req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.712445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea1f4e8f8574ff58662f84f62d36464', 'request_id': 'req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.845839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea1f4e8f8574ff58662f84f62d36464', 'request_id': 'req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.869046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] The token cache is being invalidate because project 0ea1f4e8f8574ff58662f84f62d36464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:57.869586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.870913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:57.878906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1001/1998] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:57 2022] DELETE /identity/v3/projects/0ea1f4e8f8574ff58662f84f62d36464 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:17:57.880661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:57.884931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:57.929382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870dac7-032c-463c-974e-ed7c51ae42ee service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:58.090202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:58.090735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:58.092304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:58.092304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] PATH_INFO: `/v3/users/383672bf334f4e65afbef02d227d801a` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:58.092650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=383672bf334f4e65afbef02d227d801a)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:58.094878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:58.194063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:58.196312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:58.225312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3747-be4e-4106-9c16-c4b1b1aa51f9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:58.233319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:58.234066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:58.234450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:58.234759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] PATH_INFO: `/v3/projects/2d52f9f292e24326aa80e3490cdbb370` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:58.235963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d52f9f292e24326aa80e3490cdbb370)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:58.237928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:58.731168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383672bf334f4e65afbef02d227d801a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.732074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383672bf334f4e65afbef02d227d801a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.739071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d52f9f292e24326aa80e3490cdbb370', 'request_id': 'req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.752124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d52f9f292e24326aa80e3490cdbb370', 'request_id': 'req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.761675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383672bf334f4e65afbef02d227d801a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.878110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d52f9f292e24326aa80e3490cdbb370', 'request_id': 'req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.896749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] The token cache is being invalidate because project 2d52f9f292e24326aa80e3490cdbb370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:58.897129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.897992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:58.899673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1002/1999] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:58 2022] DELETE /identity/v3/projects/2d52f9f292e24326aa80e3490cdbb370 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:58.907402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:58.907402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.001685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5b30d-f5b8-4abe-9262-279fa7cb9311 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:59.011046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:59.012253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.012701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.013092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] PATH_INFO: `/v3/projects/81d9b877ad9442c3b9f8718ef4b60ff9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.014956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d9b877ad9442c3b9f8718ef4b60ff9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:59.019454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:59.021586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383672bf334f4e65afbef02d227d801a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.038030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-12789674-20c2-40b4-89ef-28cd6e7ca78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383672bf334f4e65afbef02d227d801a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.050694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 998/2000] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:57 2022] DELETE /identity/v3/users/383672bf334f4e65afbef02d227d801a => generated 0 bytes in 1171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:59.052229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408545-43f6-4516-b2f0-d1ddf3c51537 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.052952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408545-43f6-4516-b2f0-d1ddf3c51537 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.053408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408545-43f6-4516-b2f0-d1ddf3c51537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.082359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-94408545-43f6-4516-b2f0-d1ddf3c51537 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:59.084377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-94408545-43f6-4516-b2f0-d1ddf3c51537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.212432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 999/2001] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:59.217084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.218984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.219294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.247879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d9b877ad9442c3b9f8718ef4b60ff9', 'request_id': 'req-22b103d3-c105-4206-a7ac-c100abc475dc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.250342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:17:59.257332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d9b877ad9442c3b9f8718ef4b60ff9', 'request_id': 'req-22b103d3-c105-4206-a7ac-c100abc475dc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.259586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.356669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d9b877ad9442c3b9f8718ef4b60ff9', 'request_id': 'req-22b103d3-c105-4206-a7ac-c100abc475dc'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.361786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:17:59.388404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] The token cache is being invalidate because project 81d9b877ad9442c3b9f8718ef4b60ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:17:59.388724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.390282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:17:59.395886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1003/2002] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:58 2022] DELETE /identity/v3/projects/81d9b877ad9442c3b9f8718ef4b60ff9 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:17:59.397483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1000/2003] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:17:59.400317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:59.404291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.405609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:59.422475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b50e-6d31-486e-87e1-15966170d73c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:59.423069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b103d3-c105-4206-a7ac-c100abc475dc admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:59.429744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:59.430902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.431039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:59.431665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.432014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.432357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.432900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.433157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:17:59.433581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:59.434184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] PATH_INFO: `/v3/users/0f60559bb5404f4ebc643c7b5a65459f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.436013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f60559bb5404f4ebc643c7b5a65459f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:59.436205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:59.438498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:59.459734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1001/2004] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:17:59.469427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:59.471360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.486818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa42f6a-d008-4678-821b-1425194bff45 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:59.498214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:59.510172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.510172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.510172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.510172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:59.510172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:17:59.804908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1002/2005] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:17:59 2022] POST /identity/v3/projects => generated 371 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:17:59.812163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:17:59.818022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:17:59.936866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829ba57-f3e2-4873-b7f9-099bb2a1c3e6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:17:59.945922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:17:59.946705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:17:59.959019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:17:59.959019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] PATH_INFO: `/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:17:59.959019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5bf311dc4942e6bbe8a7aa3ea200ed)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:17:59.959019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:00.000926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f60559bb5404f4ebc643c7b5a65459f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.001598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f60559bb5404f4ebc643c7b5a65459f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.018811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f60559bb5404f4ebc643c7b5a65459f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.147297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5bf311dc4942e6bbe8a7aa3ea200ed', 'request_id': 'req-bea92132-1a50-469a-88dc-cc51c23d5840'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.156614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f60559bb5404f4ebc643c7b5a65459f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.170905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f60559bb5404f4ebc643c7b5a65459f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.180932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1004/2006] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:17:59 2022] DELETE /identity/v3/users/0f60559bb5404f4ebc643c7b5a65459f => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:00.182009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:00.183660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:00.253225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5bf311dc4942e6bbe8a7aa3ea200ed', 'request_id': 'req-bea92132-1a50-469a-88dc-cc51c23d5840'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.267392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5bf311dc4942e6bbe8a7aa3ea200ed', 'request_id': 'req-bea92132-1a50-469a-88dc-cc51c23d5840'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.302399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] The token cache is being invalidate because project 5d5bf311dc4942e6bbe8a7aa3ea200ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:00.303431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.307947 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:00.315576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1003/2007] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:17:59 2022] DELETE /identity/v3/projects/5d5bf311dc4942e6bbe8a7aa3ea200ed => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:00.315576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:00.319201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:00.340525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fdff5-219f-4199-a2bd-21e24cc28807 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:00.360017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:00.360676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:00.360963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:00.361249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:00.362053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:00.364083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc077e-9edc-40ce-868c-6d27e39bda4c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:00.456906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea92132-1a50-469a-88dc-cc51c23d5840 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:00.463452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:00.465311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:00.465774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:00.466293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] PATH_INFO: `/v3/users/83622c48aaa84cdd9930b674df1791e6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:00.469408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83622c48aaa84cdd9930b674df1791e6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:00.473337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:00.743664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1005/2008] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:00 2022] POST /identity/v3/users => generated 286 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:00.745971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:00.746484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:00.746996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:00.784162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:00.786592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:00.826657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1006/2009] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:00.826657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:00.826657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:00.841825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775471b-4ede-4093-a392-9b87b918fefa None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:00.850383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:00.851072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:00.851407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:00.851698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:00.852423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:00.854111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f94ec-0331-47e0-8c99-15f4b2b605a4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:00.892740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1007/2010] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:00.904230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:00.904230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:00.904230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:00.951270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:00.953338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:00.974581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1008/2011] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:00.974581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:00.994994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd83205-c68e-4e8c-8fb2-05be7735b3c2 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:01.007831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:01.008502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.008793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.009106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.010415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:01.013687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:01.034407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1009/2012] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:01.035796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:01.061645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61331e1-b20c-49ec-b37e-a724c1719cd9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:01.072922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:01.074273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.074760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.075188 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.076798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:01.077219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:01.079471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:01.097098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1010/2013] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:01.100963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:01.125145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51666ca-6552-4894-b21a-19891e871412 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:01.133718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:01.138320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.138320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.138320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] PATH_INFO: `/v3/projects/68ed45c4bf4f4ef383577dfbd506805e/users/e69372ff650d4a94a8b9fa56d6a62d0f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.171462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83622c48aaa84cdd9930b674df1791e6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.172677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83622c48aaa84cdd9930b674df1791e6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.173833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ed45c4bf4f4ef383577dfbd506805e, user_id=e69372ff650d4a94a8b9fa56d6a62d0f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:01.194395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:01.194569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83622c48aaa84cdd9930b674df1791e6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.414108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83622c48aaa84cdd9930b674df1791e6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.423620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83622c48aaa84cdd9930b674df1791e6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.445638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1011/2014] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:01 2022] PUT /identity/v3/projects/68ed45c4bf4f4ef383577dfbd506805e/users/e69372ff650d4a94a8b9fa56d6a62d0f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:01.445638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:01.445638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:01.446416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1004/2015] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:00 2022] DELETE /identity/v3/users/83622c48aaa84cdd9930b674df1791e6 => generated 0 bytes in 1127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:01.446416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:01.446416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:01.535747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351111f9-292b-40a9-b6e9-de7755a53a24 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:01.550163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e58aa-e4d8-4be7-b38c-e9dcf4dd12e3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:01.554470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:01.554470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.554470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.554470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.556973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:01.558509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:01.559209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.559611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.560177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] PATH_INFO: `/v3/projects/45b55353ab5040c3b0adfa5288fcafec` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.561362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45b55353ab5040c3b0adfa5288fcafec)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:01.561687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7479921-3e28-403d-b743-8e1bf87e2b9f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:01.563587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:01.791003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1012/2016] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:01 2022] POST /identity/v3/projects => generated 383 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:01.792831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f16a5b-9e8f-4cbd-8bf0-471da2397c25 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:01.794045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f16a5b-9e8f-4cbd-8bf0-471da2397c25 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:01.794601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f16a5b-9e8f-4cbd-8bf0-471da2397c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:01.826264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45b55353ab5040c3b0adfa5288fcafec', 'request_id': 'req-81d48880-503f-4af3-933b-b54a2ecc928b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:01.830886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e0f16a5b-9e8f-4cbd-8bf0-471da2397c25 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:01.840915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e0f16a5b-9e8f-4cbd-8bf0-471da2397c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.001678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1013/2017] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:02.008430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.009731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.010352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.047590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45b55353ab5040c3b0adfa5288fcafec', 'request_id': 'req-81d48880-503f-4af3-933b-b54a2ecc928b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:02.050121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:02.058944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.060123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45b55353ab5040c3b0adfa5288fcafec', 'request_id': 'req-81d48880-503f-4af3-933b-b54a2ecc928b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:02.073994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1014/2018] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:02.076420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.091186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] The token cache is being invalidate because project 45b55353ab5040c3b0adfa5288fcafec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:02.092327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:02.093871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:02.096975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1005/2019] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:01 2022] DELETE /identity/v3/projects/45b55353ab5040c3b0adfa5288fcafec => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:02.104648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.108008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ef1e0-572d-40ce-afb6-c6e16ed19ae6 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.110229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.120057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.128235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.128235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.128235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.128235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.129034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.216771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d48880-503f-4af3-933b-b54a2ecc928b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.324462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1015/2020] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:02.325555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.328696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.330604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.331113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.331406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.333639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.343082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e879552-8f7a-4eb5-878f-a0c5bf9d5aad admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.346932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.347997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.348346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.348621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.349009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.350644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.355849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5a0a5-c668-4be4-a9e7-26d6d175ff60 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.364348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.367045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.368096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1006/2021] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:02.369627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a89a-870b-4ecd-be5b-0e9b99b83a33 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.370839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a89a-870b-4ecd-be5b-0e9b99b83a33 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.371488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692a89a-870b-4ecd-be5b-0e9b99b83a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.384074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1016/2022] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:02.385272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.394812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.394998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1692a89a-870b-4ecd-be5b-0e9b99b83a33 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:02.394998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1692a89a-870b-4ecd-be5b-0e9b99b83a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.404617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1007/2023] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:02.405573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c47c14-3e0a-465b-9c69-e1362a6eec3b service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.412237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.412804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.413173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.413383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.413556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.413772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.413772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.414539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.416213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.438124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1017/2024] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:02.442485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.446134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:02.463354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.463515 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88dbe3-df8d-46c3-b4cf-b45d78560665 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.463990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1008/2025] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:02.467429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.467635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.468571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.468954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.469302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.470040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.471760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.480202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0ec98-d3e0-4a4a-9a5f-c3f701343e12 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.483632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1018/2026] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:02.488009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.488927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.489183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.489722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.492038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.492038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:02.492038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.494262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.524409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12630030-8e4c-4e6f-8dc2-1491d1f2278f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.538063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.540084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.540458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.541081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] PATH_INFO: `/v3/projects/3b47a9c357b2498f9d36c1a592ad7713/users/f818c4a7386e4d5d92df1468699b8331/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.558437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1009/2027] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:02.564581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.587973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b47a9c357b2498f9d36c1a592ad7713, user_id=f818c4a7386e4d5d92df1468699b8331, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.589062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510cbf-f455-45f4-89bb-fb9f42f6d526 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.591294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528f834-b5b9-4cef-af34-e49a19dd71f3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.607581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.607930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.608125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.608452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.609188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.613972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:02.766656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1019/2028] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:02 2022] PUT /identity/v3/projects/3b47a9c357b2498f9d36c1a592ad7713/users/f818c4a7386e4d5d92df1468699b8331/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:02.772243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.772805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.773455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.800302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1010/2029] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:02.802031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:02.871606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] MFA Rules not processed for user `f818c4a7386e4d5d92df1468699b8331`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:02.872965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:02.970247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a01-2e65-47eb-b9d4-0e60f58605f8 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:02.992954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:02.994849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:02.995444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:02.995829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:02.996617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:02.998913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.039628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1020/2030] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:03.047942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:03.069036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaca162-33f7-492a-bbb9-d8bc83ef0d4a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:03.076784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:03.077572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:03.077916 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:03.078244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:03.078939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:03.081132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.260770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:03.283897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1011/2031] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:02 2022] POST /identity/v3/users => generated 293 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:03.285656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:03.309847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1021/2032] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:03 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:03.311642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f187bb-2abc-42f5-9afe-15860044c832 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:03.313433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:03.401256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:03.402027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:03.402409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:03.402829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:03.403584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:03.405404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.422793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1012/2033] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:03.422793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:03.423545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:03.440178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90b93a-6988-49a7-97b2-315cdb7688d6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:03.449526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:03.450345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:03.450782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:03.451155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] PATH_INFO: `/v3/projects/8d70052c67404d3faf82f6edb6d692ce` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:03.452424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d70052c67404d3faf82f6edb6d692ce)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:03.454921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.461796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fcb5d-f1fa-4759-9a7a-8ec0b4bc52f2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:03.477918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:03.482006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:03.482374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:03.482743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:03.483431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:03.487035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.786444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1022/2034] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:03 2022] POST /identity/v3/users => generated 298 bytes in 473 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:03.789479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:03.810903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d70052c67404d3faf82f6edb6d692ce', 'request_id': 'req-fe068e01-ed6c-4f58-bbb0-502763981b69'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:03.814065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfe557-f56c-454f-822f-433d84ad8bb9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:03.959625 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:03.961235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:03.961826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:03.962410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:03.963842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:03.967484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:03.973645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d70052c67404d3faf82f6edb6d692ce', 'request_id': 'req-fe068e01-ed6c-4f58-bbb0-502763981b69'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:03.991865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d70052c67404d3faf82f6edb6d692ce', 'request_id': 'req-fe068e01-ed6c-4f58-bbb0-502763981b69'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:03.995245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1023/2035] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:03.996169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:04.014287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a6c1-acb8-4b8b-ad70-112446c652d8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:04.015401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] The token cache is being invalidate because project 8d70052c67404d3faf82f6edb6d692ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:04.015726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.016544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.018361 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1013/2036] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:03 2022] DELETE /identity/v3/projects/8d70052c67404d3faf82f6edb6d692ce => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:04.019495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:04.023381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:04.024563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:04.025331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:04.025649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:04.025919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:04.026702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:04.028426 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:04.039436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068e01-ed6c-4f58-bbb0-502763981b69 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:04.042525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1024/2037] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:04.043971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:04.045912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:04.047060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:04.047752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:04.048055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:04.048307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] PATH_INFO: `/v3/users/a7b7684d766542b097b88d642114023b` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:04.051405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b7684d766542b097b88d642114023b)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:04.054278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:04.062548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b687cb3-4a00-46a6-aeee-1b5a72ca71ae admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:04.068425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:04.069145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:04.069491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:04.069863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] PATH_INFO: `/v3/projects/39712a615ec947aa93f1928cf9625b76/users/f7b3c7e9a0534a698a0b948a6e81120d/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:04.103686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=39712a615ec947aa93f1928cf9625b76, user_id=f7b3c7e9a0534a698a0b948a6e81120d, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:04.109058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:04.264695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1025/2038] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:04 2022] PUT /identity/v3/projects/39712a615ec947aa93f1928cf9625b76/users/f7b3c7e9a0534a698a0b948a6e81120d/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:04.265838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:04.267750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:04.365763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0d077-12c7-4f38-8d22-0b935132fe0a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:04.371695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:04.372378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:04.372666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:04.372946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] PATH_INFO: `/v3/users/de7d931aa3f04fbaac8b2e386bf2f1e5` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:04.374284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7d931aa3f04fbaac8b2e386bf2f1e5)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:04.376390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:04.415776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b7684d766542b097b88d642114023b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.416538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b7684d766542b097b88d642114023b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.430406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b7684d766542b097b88d642114023b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.703703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b7684d766542b097b88d642114023b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.713056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b7684d766542b097b88d642114023b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.720303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1014/2039] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:04 2022] DELETE /identity/v3/users/a7b7684d766542b097b88d642114023b => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:04.721714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:04.738043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:04.827373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a093f3-0b14-40fa-b546-6ee75f31d241 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:04.835539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:04.836709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:04.837212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:04.837709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] PATH_INFO: `/v3/projects/d19abb22acc144b083eb782663bbe60b/users/c42e953a6ad144859383f6f94dd67015/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:04.865168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19abb22acc144b083eb782663bbe60b, user_id=c42e953a6ad144859383f6f94dd67015, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:04.870894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac3e7a-1cc1-4797-840a-d4dc6a0da329 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:04.889587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7d931aa3f04fbaac8b2e386bf2f1e5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.895927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7d931aa3f04fbaac8b2e386bf2f1e5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:04.910237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7d931aa3f04fbaac8b2e386bf2f1e5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.113922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7d931aa3f04fbaac8b2e386bf2f1e5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.123461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7d931aa3f04fbaac8b2e386bf2f1e5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.123800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1015/2040] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:04 2022] PUT /identity/v3/projects/d19abb22acc144b083eb782663bbe60b/users/c42e953a6ad144859383f6f94dd67015/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:05.125930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:05.126409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:05.126914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:05.131894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1026/2041] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:04 2022] DELETE /identity/v3/users/de7d931aa3f04fbaac8b2e386bf2f1e5 => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:05.132902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:05.134223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:05.172871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] MFA Rules not processed for user `f7b3c7e9a0534a698a0b948a6e81120d`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:05.174170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:05.224924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35095d-154f-4a76-865a-42022921df86 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:05.234821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:05.235833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:05.236684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:05.236891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] PATH_INFO: `/v3/users/d191f7381b2e48ab979fcdb672b9e33f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:05.239175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d191f7381b2e48ab979fcdb672b9e33f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:05.244301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:05.290821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1016/2042] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:05.291917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:05.293980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:05.317689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74472f-5b5d-42b5-b08f-23c55e3371c9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:05.326885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:05.326885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:05.327371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:05.327844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:05.329267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:05.332364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:05.356088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1017/2043] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:05.356088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:05.356088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:05.372798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ea2a-c569-4dcc-aa0b-3c8b72d47f65 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:05.384679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:05.385483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:05.385863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:05.386428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] PATH_INFO: `/v3/projects/bf6936f23aa446008e8119807ec8724d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:05.387930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6936f23aa446008e8119807ec8724d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:05.392076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:05.628137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd191f7381b2e48ab979fcdb672b9e33f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.629787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd191f7381b2e48ab979fcdb672b9e33f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.657102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd191f7381b2e48ab979fcdb672b9e33f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.763552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6936f23aa446008e8119807ec8724d', 'request_id': 'req-9c4c4679-e1ab-4836-9eef-18eaee2e013d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.839404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd191f7381b2e48ab979fcdb672b9e33f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.847966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd191f7381b2e48ab979fcdb672b9e33f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.857360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1027/2044] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:05 2022] DELETE /identity/v3/users/d191f7381b2e48ab979fcdb672b9e33f => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:05.858238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:05.863788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:05.946458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e40b76-82aa-4f99-8ee1-8682eaea5555 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:05.951486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:05.952097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:05.952376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:05.952689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:05.953316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:05.955304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a64f4d-28ae-408f-af6b-6c98742d7561 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:05.961358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6936f23aa446008e8119807ec8724d', 'request_id': 'req-9c4c4679-e1ab-4836-9eef-18eaee2e013d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.969320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6936f23aa446008e8119807ec8724d', 'request_id': 'req-9c4c4679-e1ab-4836-9eef-18eaee2e013d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.988849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] The token cache is being invalidate because project bf6936f23aa446008e8119807ec8724d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:05.989238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.990274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:05.992672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1018/2045] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:05 2022] DELETE /identity/v3/projects/bf6936f23aa446008e8119807ec8724d => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:05.993849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:05.995684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.069813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c4679-e1ab-4836-9eef-18eaee2e013d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:06.074719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:06.075355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.075639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.075914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] PATH_INFO: `/v3/projects/d19abb22acc144b083eb782663bbe60b/users/c42e953a6ad144859383f6f94dd67015/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.078150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19abb22acc144b083eb782663bbe60b, user_id=c42e953a6ad144859383f6f94dd67015, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:06.080885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71afc2-7dd4-4324-827e-039403ea46e4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:06.095472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1019/2046] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:05 2022] PUT /identity/v3/projects/d19abb22acc144b083eb782663bbe60b/users/c42e953a6ad144859383f6f94dd67015/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:06.097208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.097671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.098094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.133133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] MFA Rules not processed for user `e69372ff650d4a94a8b9fa56d6a62d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:06.134375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.199079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1028/2047] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:05 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:06.201370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cc68-cbfa-4e8b-bf04-52cfc65b0a35 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.201811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cc68-cbfa-4e8b-bf04-52cfc65b0a35 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.202873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2cc68-cbfa-4e8b-bf04-52cfc65b0a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.228823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1020/2048] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:06.229847 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:06.231805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.234517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-74c2cc68-cbfa-4e8b-bf04-52cfc65b0a35 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:06.236275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-74c2cc68-cbfa-4e8b-bf04-52cfc65b0a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.323875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1029/2049] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:06.324059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ea3c2-7a59-4f34-949c-bf90da10e282 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:06.324253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627608-e7e6-428c-8035-bee9068b7fa6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.324253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627608-e7e6-428c-8035-bee9068b7fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.324253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66627608-e7e6-428c-8035-bee9068b7fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.326933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:06.326933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.326933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.328251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] PATH_INFO: `/v3/projects/ad240654f1134efe941f54ff577768a5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.328658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad240654f1134efe941f54ff577768a5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:06.330550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:06.358288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-66627608-e7e6-428c-8035-bee9068b7fa6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:06.359592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-66627608-e7e6-428c-8035-bee9068b7fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.372675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1030/2050] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:06.374327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.374774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.375139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.413049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] MFA Rules not processed for user `c42e953a6ad144859383f6f94dd67015`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:06.415265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.491935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1031/2051] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:06.493121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:06.494868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.515096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd608b00-f388-4e22-bae0-a17f07928456 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:06.522909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:06.523529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:06.523816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:06.524094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:06.524727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:06.526689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:06.814216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad240654f1134efe941f54ff577768a5', 'request_id': 'req-0faceab1-fd6b-4720-909b-2b52516bda6e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:06.911296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1032/2052] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/users => generated 299 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:06.912842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:06.914869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:06.928718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d580819-dad8-41e1-9f1d-ab7c280d4a4f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.001230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.001928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.002102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.002381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.003057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.005103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240228b-2aa3-4f0a-ae5e-169e126857c4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.113468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad240654f1134efe941f54ff577768a5', 'request_id': 'req-0faceab1-fd6b-4720-909b-2b52516bda6e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:07.119979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad240654f1134efe941f54ff577768a5', 'request_id': 'req-0faceab1-fd6b-4720-909b-2b52516bda6e'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:07.133319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] The token cache is being invalidate because project ad240654f1134efe941f54ff577768a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:07.133607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:07.137549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:07.137549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1021/2053] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:06 2022] DELETE /identity/v3/projects/ad240654f1134efe941f54ff577768a5 => generated 0 bytes in 906 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:07.137549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.138059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.207372 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faceab1-fd6b-4720-909b-2b52516bda6e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.212011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1033/2054] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:06 2022] POST /identity/v3/projects => generated 371 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:07.213501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.213878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.214200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.219862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.219988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.219988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.219988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.219988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.232848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.234380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.244646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:07.245282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1022/2055] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:07.245884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.246180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.247792 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.340407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:07.345285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa6116-fb5f-4aef-9110-40a51465ecdd service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.371006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1034/2056] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:07.372671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.375604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.400756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fcd0b-3dcf-419b-97fb-8651c116c025 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.410702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.411822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.413479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.413950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.414747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.417853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcdc432-afd7-4221-b434-1a204aa71d41 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.425432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.427017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.427608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.429608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.432180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.446836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1035/2057] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:07.449445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.450125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.450430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.474167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:07.475801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.489270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1036/2058] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:07.490992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.493187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.508407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a12739-688c-47b0-bade-8d56e14efd6b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.517321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.517960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.518305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.518587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.519282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.521989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961e4bc-b484-4424-915b-4ae64d7df25c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.549310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.550983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.564484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1023/2059] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:07.567190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.580620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ca2ae-39f0-49ca-a295-71f42dea3300 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.591498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.603678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.612518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1024/2060] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:07.613515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.635148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11014c6-beef-492d-ac18-b76a25052fed admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.635148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.635148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.635148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.635523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] PATH_INFO: `/v3/projects/fa060dbc686a4ccfa87e95a1a535261f/users/36473b1ad30f4833a5cc35f523613adb/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.656352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa060dbc686a4ccfa87e95a1a535261f, user_id=36473b1ad30f4833a5cc35f523613adb, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.659506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:07.935737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1037/2061] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:07 2022] POST /identity/v3/users => generated 292 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:07.943068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.943068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.943068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.957470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:07.958740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:07.967613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1038/2062] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:07.968404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:07.978569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c270880-a95d-433a-90c1-e87c5dc0ec67 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:07.984160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:07.984782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:07.985090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:07.985391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:07.986123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:07.988115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.048956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1025/2063] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:07 2022] PUT /identity/v3/projects/fa060dbc686a4ccfa87e95a1a535261f/users/36473b1ad30f4833a5cc35f523613adb/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:08.049890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.121016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfc4bd-1e06-4575-b56f-799dd2cad452 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.129123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.133782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.134057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.134870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.136548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:08.210935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.212404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.218959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1039/2064] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:07 2022] POST /identity/v3/projects => generated 389 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:08.221248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.225984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1026/2065] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:08.228131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.267991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:08.328960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b455f-f53c-4964-b94b-64646d3875a7 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.344575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac20efa-c4e3-4d74-9b04-c675b699513a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.353330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.354309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.354987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.355472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.356590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.359421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.378675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1040/2066] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:08.378675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.390739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380682-e8b4-4a30-9e7e-b5ae331a0be6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.401567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.402538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.403197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.403656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.404771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.408394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.425990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1041/2067] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:08.428145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.444656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3d468-feb1-4dde-8192-f984a3c113ca admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.447921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.448995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.449474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.449935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.450986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:08.451632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.452831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.453801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.454230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.454556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.454724 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.455731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.459576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.473904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1027/2068] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:08.474845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.485050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4f1ce-ece2-467c-a15e-2a7b110cb873 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.489775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.490999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.491285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.491564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] PATH_INFO: `/v3/projects/8d9d9069e3e543a98ef75b2461cf3242/users/0c914c46614a4495ab06e7af013f683c/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.508558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9d9069e3e543a98ef75b2461cf3242, user_id=0c914c46614a4495ab06e7af013f683c, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.511305 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.790751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1042/2069] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:08 2022] POST /identity/v3/users => generated 295 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:08.790751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.817047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b0e35-f6b4-4e89-a9fa-a07adefe5d9f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.817047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.817047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.817047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.817596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] PATH_INFO: `/v3/projects/fa060dbc686a4ccfa87e95a1a535261f/users/36473b1ad30f4833a5cc35f523613adb/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.817720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa060dbc686a4ccfa87e95a1a535261f, user_id=36473b1ad30f4833a5cc35f523613adb, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.820553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:08.835990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1043/2070] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:08 2022] PUT /identity/v3/projects/fa060dbc686a4ccfa87e95a1a535261f/users/36473b1ad30f4833a5cc35f523613adb/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:08.837047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.860923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1028/2071] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:08 2022] PUT /identity/v3/projects/8d9d9069e3e543a98ef75b2461cf3242/users/0c914c46614a4495ab06e7af013f683c/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:08.861530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:08.865971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:08.931469 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8321ae-38e5-4035-bd6b-499d38b88c41 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.963872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c9e6c-48dd-4b9f-80e9-d473e7fddf3d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:08.970560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:08.971298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:08.971602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:08.971931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] PATH_INFO: `/v3/projects/9930af8bcf7f4bdbb00ee6b3a9ca22db` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:08.973277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9930af8bcf7f4bdbb00ee6b3a9ca22db)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:08.975747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.011739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.012716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.013187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.013640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.014513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.016827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.294788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9930af8bcf7f4bdbb00ee6b3a9ca22db', 'request_id': 'req-ec75aee5-a937-45ee-a98c-7d923fa1d14b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:09.367286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1044/2072] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:08 2022] POST /identity/v3/projects => generated 377 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:09.368673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.444043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a4208-5127-4456-87b6-7af801421f14 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.448969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.449601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.449886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.450164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.450965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.452618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456991ab-e209-4d06-ba55-ec12c3bcda69 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.463502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1045/2073] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:09.464875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.465174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.465462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.495482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] MFA Rules not processed for user `36473b1ad30f4833a5cc35f523613adb`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:09.496732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.588401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1046/2074] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:09.589177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.603073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c68f9-436e-4a1c-b44e-68c5f241376b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.608626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.609285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.609539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.609819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.610476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.612146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.629928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9930af8bcf7f4bdbb00ee6b3a9ca22db', 'request_id': 'req-ec75aee5-a937-45ee-a98c-7d923fa1d14b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:09.629928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9930af8bcf7f4bdbb00ee6b3a9ca22db', 'request_id': 'req-ec75aee5-a937-45ee-a98c-7d923fa1d14b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:09.630253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1047/2075] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:09.630253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.636631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ab680-819f-4122-b495-a95755666408 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.640265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] The token cache is being invalidate because project 9930af8bcf7f4bdbb00ee6b3a9ca22db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:09.640639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:09.641396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:09.643136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1029/2076] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:08 2022] DELETE /identity/v3/projects/9930af8bcf7f4bdbb00ee6b3a9ca22db => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:09.643510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.644722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.646955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.646955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.650014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.654534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.655340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.655597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363430e1-428c-4ae3-8567-3365a3fbdc58 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.728098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75aee5-a937-45ee-a98c-7d923fa1d14b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.733549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.734196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.734481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.734808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.735462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.737081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.748917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1030/2077] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:09.749781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.751552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.827646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3363f03-ae15-4383-9120-e16f46744358 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.831392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.831996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.832275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.832542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.835215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.858993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.858993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.870771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1031/2078] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:09.871662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:09.872985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.889620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c54e-a895-4530-904e-1918bbb4490d service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:09.906700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:09.906700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.907503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1048/2079] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:09 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:09.909561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:09.909561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.909561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.920426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:09.921065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] PATH_INFO: `/v3/projects/8d9d9069e3e543a98ef75b2461cf3242/users/0c914c46614a4495ab06e7af013f683c/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:09.924634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9d9069e3e543a98ef75b2461cf3242, user_id=0c914c46614a4495ab06e7af013f683c, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:09.930532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:09.947989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:09.953963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:09.954110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1032/2080] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:09 2022] PUT /identity/v3/projects/8d9d9069e3e543a98ef75b2461cf3242/users/0c914c46614a4495ab06e7af013f683c/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:09.954110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.076206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1049/2081] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:10.077375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.080835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.081676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c7735-1d5a-40cc-b71e-e6bced39ad87 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.087098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.087798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.088078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.088353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] PATH_INFO: `/v3/projects/f7e0c375a58842e18a4306779fd6e167/users/660c6df77d0a4750aac30345d6768526/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.103628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11def488-c67e-46e4-80cc-f1b236858beb None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.107301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.108048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.108470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.108749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] PATH_INFO: `/v3/users/f7b3c7e9a0534a698a0b948a6e81120d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.110052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7b3c7e9a0534a698a0b948a6e81120d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.112190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.118542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e0c375a58842e18a4306779fd6e167, user_id=660c6df77d0a4750aac30345d6768526, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.126035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.245158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1033/2082] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:09 2022] PUT /identity/v3/projects/f7e0c375a58842e18a4306779fd6e167/users/660c6df77d0a4750aac30345d6768526/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:10.246437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.248842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.367471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0204-48a4-479b-86f5-107ccf2f404c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.371347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.372076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.372365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.372640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.373306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.376065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be508a-d5a3-48cd-aec3-1382b1529454 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.388105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1034/2083] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:10.389520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62479488-64fc-4dbf-9734-3be36317656f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.394919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62479488-64fc-4dbf-9734-3be36317656f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.394919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62479488-64fc-4dbf-9734-3be36317656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.407368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:10.412462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-62479488-64fc-4dbf-9734-3be36317656f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:10.414178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-62479488-64fc-4dbf-9734-3be36317656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.426029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1035/2084] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:10.427358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.428033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.428320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.452447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:10.453682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.462732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1036/2085] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:10.463805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.477111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a4ccc-41b3-409b-b948-4538a15ae337 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.482201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.482845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.483130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.483406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.484072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.485802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.500050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1037/2086] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:10.500520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.518125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7b3c7e9a0534a698a0b948a6e81120d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:10.519078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7b3c7e9a0534a698a0b948a6e81120d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:10.522289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f907e7-8d62-4e33-a4d6-61980b854e1f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.536181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7b3c7e9a0534a698a0b948a6e81120d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:10.541636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.542733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.543640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.544202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.544913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:10.545827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.548005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.581627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1038/2087] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:10.581627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.584193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.653155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7b3c7e9a0534a698a0b948a6e81120d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:10.667579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7b3c7e9a0534a698a0b948a6e81120d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:10.675763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1050/2088] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:10 2022] DELETE /identity/v3/users/f7b3c7e9a0534a698a0b948a6e81120d => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:10.679818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.681609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.686511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd769d-321f-4cc4-9cd5-d6207da1a62e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.761675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d816e5-3ad3-4018-95be-ce0ae8eba1a3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.766705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.767570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.767796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.768079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] PATH_INFO: `/v3/projects/a594c7ea93d943cdaf9d8205dfec08df/users/12b490db1ec948c69206001ed11da8a9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.769310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.770026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.770377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.770688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.773555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.788080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a594c7ea93d943cdaf9d8205dfec08df, user_id=12b490db1ec948c69206001ed11da8a9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.789921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.791018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8993336-8386-4bd2-b116-00664ed9d940 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.791495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:10.808936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1039/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:10.809790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:10.811176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:10.826081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b798a84-f9a5-41ca-b585-23bba575ebd5 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:10.833856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:10.834490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:10.834818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:10.835119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:10.835831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:10.837979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.038583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1051/2090] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:10 2022] PUT /identity/v3/projects/a594c7ea93d943cdaf9d8205dfec08df/users/12b490db1ec948c69206001ed11da8a9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:11.040894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1040/2091] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:10 2022] POST /identity/v3/projects => generated 381 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:11.041572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9de-2e91-4931-8e44-1ff64f1d440e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.042067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9de-2e91-4931-8e44-1ff64f1d440e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.042545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe9de-2e91-4931-8e44-1ff64f1d440e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.052347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.081976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ffefe9de-2e91-4931-8e44-1ff64f1d440e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:11.086245 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ffefe9de-2e91-4931-8e44-1ff64f1d440e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:11.178665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ebd59-8099-4c9c-be97-88fbb4c66e51 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.183513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1052/2092] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:11.185812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.186289 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.187292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.188852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.189493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.189797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.190132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.190811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.192977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.228505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] MFA Rules not processed for user `12b490db1ec948c69206001ed11da8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:11.229809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:11.338129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1041/2093] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:11 2022] POST /identity/v3/users => generated 291 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:11.339765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.346468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:11.352718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1053/2094] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:11.353773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.373605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282520eb-6c17-4f4b-b244-64d45f2d1d70 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.384847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.386320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.386320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.386873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.389203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.393304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.425408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1054/2095] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:11.425408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.436239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:11.515158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7191a-dd93-4662-860a-ca0a60c3a6cc admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.523350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.524656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.529298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.529298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.569041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.571959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.587859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b78f68-9fe1-4cf4-bd5c-5b4caba175cd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.592570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.592842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1042/2096] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:11.593359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.593748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.593915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.594285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.611557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.621239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.625197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e56-ea99-4576-9b2c-babfd51b2aac service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.634902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.635539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1055/2097] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:11.637625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.637625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.637625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.638270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.641533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beced9aa-9a30-4240-a09d-8c4ff13d1cf0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.671531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1043/2098] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:11.676027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:11.700915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184c550-7cc1-4b9b-895e-e13b3e9dc5ff service glance] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:11.710706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:11.711975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.712558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.713038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] PATH_INFO: `/v3/projects/7e90dec3a44a4763b0045af64212fc15/users/543a9cddd92a44c78c26e5b8400e0c98/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.768072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:11.775040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e90dec3a44a4763b0045af64212fc15, user_id=543a9cddd92a44c78c26e5b8400e0c98, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:11.797382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:11.964927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1056/2099] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:11 2022] PUT /identity/v3/projects/7e90dec3a44a4763b0045af64212fc15/users/543a9cddd92a44c78c26e5b8400e0c98/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:11.970922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:11.971393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:11.971814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:11.999135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:12.000369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:12.138737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1044/2100] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:12.161386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:12.270176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e269747-4051-43bf-96e6-f30c52151a22 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:12.275744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:12.275945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:12.276213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:12.276716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:12.286663 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:12.288896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08c10a-ed47-4dd2-bfa1-9b3dac68d361 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:12.308382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1057/2101] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:12.470815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:12.531298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:12.538141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:12.538785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:12.538785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:12.538785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:12.553240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf9c65-5828-4877-94d8-73707b0e5d2d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:12.559342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:12.560035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:12.566482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:12.566482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] PATH_INFO: `/v3/projects/39712a615ec947aa93f1928cf9625b76` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:12.566482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39712a615ec947aa93f1928cf9625b76)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:12.566482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:12.593517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] MFA Rules not processed for user `0c914c46614a4495ab06e7af013f683c`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:12.594890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:12.694236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1058/2102] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:12.727937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:12.730000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:12.761563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39712a615ec947aa93f1928cf9625b76', 'request_id': 'req-027b4cc0-7c17-4c53-9103-d99eb0613b84'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:12.833143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e3e3b-5c62-4a8a-b9c7-ab88c41a78d7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:12.897118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:12.897771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:12.898053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:12.898326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:12.900165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:12.928089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39712a615ec947aa93f1928cf9625b76', 'request_id': 'req-027b4cc0-7c17-4c53-9103-d99eb0613b84'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:12.937653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39712a615ec947aa93f1928cf9625b76', 'request_id': 'req-027b4cc0-7c17-4c53-9103-d99eb0613b84'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:12.958382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] The token cache is being invalidate because project 39712a615ec947aa93f1928cf9625b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:12.960281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:12.961338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:12.963547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1045/2103] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:12 2022] DELETE /identity/v3/projects/39712a615ec947aa93f1928cf9625b76 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:12.969238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:12.973091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:13.018975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:13.020544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:13.044509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1059/2104] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:13.144636 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b4cc0-7c17-4c53-9103-d99eb0613b84 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:13.153238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:13.154770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:13.155883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:13.156051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] PATH_INFO: `/v3/users/36473b1ad30f4833a5cc35f523613adb` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:13.157760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36473b1ad30f4833a5cc35f523613adb)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:13.164109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:13.171674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:13.173409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:13.197366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2becc05-dc40-4c5b-af18-323bf0e5852b service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:13.210179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:13.211456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:13.212162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:13.212696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] PATH_INFO: `/v3/users/f818c4a7386e4d5d92df1468699b8331` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:13.220472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f818c4a7386e4d5d92df1468699b8331)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:13.224169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:13.704049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36473b1ad30f4833a5cc35f523613adb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.705787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36473b1ad30f4833a5cc35f523613adb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.709815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f818c4a7386e4d5d92df1468699b8331'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.710986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f818c4a7386e4d5d92df1468699b8331'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.733164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36473b1ad30f4833a5cc35f523613adb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.735634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f818c4a7386e4d5d92df1468699b8331'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.857437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36473b1ad30f4833a5cc35f523613adb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.868512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-dab8d4b2-2a30-42f8-a255-d79d83ce4512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36473b1ad30f4833a5cc35f523613adb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.884858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1046/2105] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:12 2022] DELETE /identity/v3/users/36473b1ad30f4833a5cc35f523613adb => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:13.969386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f818c4a7386e4d5d92df1468699b8331'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.981375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f818c4a7386e4d5d92df1468699b8331'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:13.995957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1060/2106] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:13 2022] DELETE /identity/v3/users/f818c4a7386e4d5d92df1468699b8331 => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:14.001665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:14.002035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:14.002491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:14.035152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:14.037402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:14.128139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1047/2107] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:14.143567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:14.145673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:14.243689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c456e-48cd-441f-9695-3374c7a240b9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:14.259215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:14.259215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:14.259913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:14.260108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:14.274875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:14.276569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51b6c-e882-47d9-9894-c48775e6754e service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:14.286035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:14.296587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1061/2108] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:14.727073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:14.728721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:14.743461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71dc30-af3e-43eb-a421-f57ebc9baed5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:14.758401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:14.759174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:14.759459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:14.759732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] PATH_INFO: `/v3/projects/fa060dbc686a4ccfa87e95a1a535261f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:14.763093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa060dbc686a4ccfa87e95a1a535261f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:14.770248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:14.992818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa060dbc686a4ccfa87e95a1a535261f', 'request_id': 'req-19cb4516-b4d8-42e0-ab3a-131a31064b1c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.156688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa060dbc686a4ccfa87e95a1a535261f', 'request_id': 'req-19cb4516-b4d8-42e0-ab3a-131a31064b1c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.165359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa060dbc686a4ccfa87e95a1a535261f', 'request_id': 'req-19cb4516-b4d8-42e0-ab3a-131a31064b1c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.186665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] The token cache is being invalidate because project fa060dbc686a4ccfa87e95a1a535261f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:15.186975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.187904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.190679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1048/2109] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:14 2022] DELETE /identity/v3/projects/fa060dbc686a4ccfa87e95a1a535261f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:15.197824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:15.198360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:15.198676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:15.218981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:15.223339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.252819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:15.254702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.421186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1062/2110] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:15.432886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:15.440756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:15.440965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.447109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb4516-b4d8-42e0-ab3a-131a31064b1c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:15.456200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:15.457296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:15.457890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:15.458288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] PATH_INFO: `/v3/projects/3b47a9c357b2498f9d36c1a592ad7713` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:15.460317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b47a9c357b2498f9d36c1a592ad7713)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:15.463232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:15.561642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16a82-3ae0-4137-8cf4-ff0f9a068677 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:15.580587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:15.581632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:15.581980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:15.582312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:15.586215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.632517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b47a9c357b2498f9d36c1a592ad7713', 'request_id': 'req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.716692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:15.718256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e932a-8ab5-4be0-96e4-d599c2b9a729 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:15.735534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1063/2111] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:15.737984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:15.737984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:15.737984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:15.752051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b47a9c357b2498f9d36c1a592ad7713', 'request_id': 'req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.763353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b47a9c357b2498f9d36c1a592ad7713', 'request_id': 'req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.777423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:15.779119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.779550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] The token cache is being invalidate because project 3b47a9c357b2498f9d36c1a592ad7713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:15.779956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.785662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:15.791276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1049/2112] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:15 2022] DELETE /identity/v3/projects/3b47a9c357b2498f9d36c1a592ad7713 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:15.792968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:15.797565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:15.975794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1064/2113] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:15.976820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:16.001713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45a0e2-9c2f-4abc-b4fa-8a11b59bc013 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:16.008693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:16.009588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.009951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.010381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.011926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:16.012311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:16.016225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1da78a-3a2c-4dfb-bb64-d86fb9d30d4e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:16.025959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6948f-466f-40f1-ad20-e5d4c14a2a2a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:16.045178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1065/2114] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:16.051285 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.052018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:16.053165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.053986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.054495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.055209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] PATH_INFO: `/v3/users/c42e953a6ad144859383f6f94dd67015` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.055961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.057346 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42e953a6ad144859383f6f94dd67015)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:16.060089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:16.135764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] MFA Rules not processed for user `543a9cddd92a44c78c26e5b8400e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:16.137777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:16.278559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1066/2115] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:16.279999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:16.295863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862d5ca-f23b-4c8f-acb1-d704ff9b2f56 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:16.305969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:16.307100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.307616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.308013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.309205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:16.312419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:16.405900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42e953a6ad144859383f6f94dd67015'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:16.406994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42e953a6ad144859383f6f94dd67015'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:16.429776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42e953a6ad144859383f6f94dd67015'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:16.479144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1067/2116] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:16 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:16.479144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:16.479144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:16.624365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42e953a6ad144859383f6f94dd67015'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:16.659633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42e953a6ad144859383f6f94dd67015'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:16.670554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dadc8d-c328-4295-a7ee-e0c9870e9192 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:16.675116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:16.675892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.676298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.676681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.679424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:16.681954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1050/2117] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:15 2022] DELETE /identity/v3/users/c42e953a6ad144859383f6f94dd67015 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:16.683724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:16.710023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:16.820427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:16.822306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:16.847252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da413919-47ca-4093-922d-021cd4781230 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:16.855027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:16.855989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:16.856423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:16.856849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:16.858052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:16.861819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:16.863842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1068/2118] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:17.012449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1051/2119] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:16 2022] POST /identity/v3/users => generated 290 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:17.018304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:17.141292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5f4c-7ba3-44ef-868c-fbf1d6cf3e01 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:17.149746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:17.150820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:17.151300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:17.151851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:17.153049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:17.156234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:17.179892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1069/2120] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:17.181118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:17.198451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d840-4ef6-4e77-a60e-4ef5ef779e62 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:17.206424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:17.207459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:17.207933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:17.208392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:17.210146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:17.213107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:17.235150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1052/2121] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:17.242896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:17.262169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909bf59-545b-47ec-b46c-09d2614baa75 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:17.275924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:17.276776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:17.277457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:17.277893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] PATH_INFO: `/v3/projects/9cd8e6a3637a4db8b88535d0e1bd81f4/users/60f5b19f74da48ec8466d334ab87da24/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:17.305170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd8e6a3637a4db8b88535d0e1bd81f4, user_id=60f5b19f74da48ec8466d334ab87da24, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:17.308556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6860d-e052-4e19-af7c-4be44be5baca admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:17.754857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:17.757618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:17.771782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1070/2122] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:17 2022] PUT /identity/v3/projects/9cd8e6a3637a4db8b88535d0e1bd81f4/users/60f5b19f74da48ec8466d334ab87da24/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:17.785912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:17.785912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:17.785912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:17.791101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90469f2-afc7-4c4e-aa5a-5d7cf25a5ac3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:17.873244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] MFA Rules not processed for user `60f5b19f74da48ec8466d334ab87da24`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:17.877170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:18.014608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.015772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.016935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.019857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] PATH_INFO: `/v3/projects/d19abb22acc144b083eb782663bbe60b` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.023784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19abb22acc144b083eb782663bbe60b)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.033717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.123572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1071/2123] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:18.128405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.141006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94f3c5-3f02-4425-b63f-ac4539b4f70b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.151565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.152267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.152653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.153035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.153843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.157756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.298820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19abb22acc144b083eb782663bbe60b', 'request_id': 'req-e8823dd0-db21-4ae3-b991-0a9132ffc188'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:18.349392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1072/2124] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:18 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:18.357299 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.507415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19abb22acc144b083eb782663bbe60b', 'request_id': 'req-e8823dd0-db21-4ae3-b991-0a9132ffc188'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:18.514449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a3da5-da51-4bd3-a0a7-914c03851f5a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.516676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19abb22acc144b083eb782663bbe60b', 'request_id': 'req-e8823dd0-db21-4ae3-b991-0a9132ffc188'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:18.526428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.527224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.527567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.527953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.530123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.536136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.546658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] The token cache is being invalidate because project d19abb22acc144b083eb782663bbe60b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:18.548775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:18.550409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e8823dd0-db21-4ae3-b991-0a9132ffc188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:18.554996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1053/2125] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:17 2022] DELETE /identity/v3/projects/d19abb22acc144b083eb782663bbe60b => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:18.560754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34e675-3a4d-47b9-8b56-66d838be0bd0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.561044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34e675-3a4d-47b9-8b56-66d838be0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.561323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34e675-3a4d-47b9-8b56-66d838be0bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.598811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9e34e675-3a4d-47b9-8b56-66d838be0bd0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:18.602257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9e34e675-3a4d-47b9-8b56-66d838be0bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:18.618435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1054/2126] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:18.628428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.629567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.629976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.657185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:18.659454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:18.675284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1055/2127] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:18.676502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.689890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2938dc5-ac03-4905-a93d-8c0c7063f205 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.705783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1073/2128] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:18 2022] POST /identity/v3/users => generated 289 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:18.712323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.714903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.716090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.717646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.718268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.718580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:18.720276 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:18.723319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.726107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.748839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c7ca6-48e2-48f7-b8f3-a6ae106ce9bd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.772350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.772973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.773267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.773549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.774256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.776062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.781161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1056/2129] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:18.785185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.818996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1074/2130] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:18.820368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.820539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6205e9-352e-4774-a6f7-1b9fa58ffeb4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.832898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.834145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.839309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.839309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.839309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.839309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.847018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2bb91-3d20-4b59-bf17-3f0c71778359 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.868531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.871481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.871935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.872331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.873142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.875655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:18.891412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1075/2131] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:18.899538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:18.914444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4b0bd-0f78-404b-96bd-a05720f1864c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:18.921850 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:18.922994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:18.923479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:18.923901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] PATH_INFO: `/v3/projects/92b36f73655b4221a23e6c302f3518ad/users/0e875c9d84cf4b26859699572d689d7f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:18.979900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b36f73655b4221a23e6c302f3518ad, user_id=0e875c9d84cf4b26859699572d689d7f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:18.988700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002acdee-78b9-4013-8232-e68219cf7957 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.049646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1057/2132] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:18 2022] POST /identity/v3/projects => generated 371 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:19.057091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:19.175350 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1076/2133] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:18 2022] PUT /identity/v3/projects/92b36f73655b4221a23e6c302f3518ad/users/0e875c9d84cf4b26859699572d689d7f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:19.194335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.194589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.196619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.220275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e428f1-94bf-4a54-929d-b4c8a71549f8 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:19.264620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] MFA Rules not processed for user `0e875c9d84cf4b26859699572d689d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:19.266201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:19.365191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:19.366981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.367574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.367930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.369894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:19.375922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.418953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1077/2134] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:19.418953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:19.420997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b04d6a-e65d-4fdf-a18a-54b04f4f3c7d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:19.430908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:19.431716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.432086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.432438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.434662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:19.436648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.634094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1058/2135] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:19 2022] POST /identity/v3/users => generated 286 bytes in 579 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:19.644064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:19.669446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726a6ac-a47d-48d6-a060-33523a996cfb admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:19.674984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:19.675661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.675941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.676212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.676870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:19.679321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98603b-5188-467b-ab31-db153ef83ff5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.692310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1059/2136] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:19.695991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.695991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.695991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.720494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] MFA Rules not processed for user `543a9cddd92a44c78c26e5b8400e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:19.721780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:19.795653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1078/2137] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:19 2022] POST /identity/v3/projects => generated 379 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:19.797823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1060/2138] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:19.798445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:19.801672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:19.815759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbc84e-23e3-41be-ac14-9f12d96dbbcc None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:19.906461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:19.907267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.910350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.911055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.912885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:19.928519 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.953433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a99e5-096d-4997-9742-29d18288edcf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:19.963911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:19.964932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:19.965552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:19.966037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:19.967101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:19.969217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:19.983224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1079/2139] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:19.991695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.006292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910253c-52e6-41e5-a3e6-1db11f2e2140 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.012934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.014589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.015023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.015465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] PATH_INFO: `/v3/projects/248f8afef2a14e12a6eafaf644d50da0/users/59d04da51cab4d43a8d9bafe0b6560f1/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.042358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1061/2140] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:19 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:20.047860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.055478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=248f8afef2a14e12a6eafaf644d50da0, user_id=59d04da51cab4d43a8d9bafe0b6560f1, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.061791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ebde2-d4a8-4e76-8fac-8d278dddc7fe admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.072824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6befa6-572e-4681-9020-9b6a6e3f72f9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.085924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.087570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.088287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.089246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.091143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.095209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.130414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1062/2141] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:20.136414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.151045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38b5cc-08d6-4844-bf42-5a8558a53d4a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.160338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.164839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.165468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.165850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] PATH_INFO: `/v3/projects/afc0d5e8b6554ad0821e999888ade90b/users/0995a910405849e693e2e9b496d8a646/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.225582 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1080/2142] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:19 2022] PUT /identity/v3/projects/248f8afef2a14e12a6eafaf644d50da0/users/59d04da51cab4d43a8d9bafe0b6560f1/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:20.241664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.242363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.242912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.252069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc0d5e8b6554ad0821e999888ade90b, user_id=0995a910405849e693e2e9b496d8a646, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.254982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.307871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] MFA Rules not processed for user `59d04da51cab4d43a8d9bafe0b6560f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:20.309170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:20.456987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1063/2143] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:20 2022] PUT /identity/v3/projects/afc0d5e8b6554ad0821e999888ade90b/users/0995a910405849e693e2e9b496d8a646/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:20.465253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.465461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1081/2144] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:20.511333 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.514747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:20.652439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc5727-c352-4161-b3bf-3cc18b478fcb admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.674777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99d25c-2ced-455b-b133-71b3f6adea04 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.680862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.681513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.681796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.682077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.682887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.695109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.734510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1064/2145] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:20.739165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.773657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69410604-d6f9-48bc-96dc-661cdc978df9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:20.796773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.798477 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:20.799149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.799429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.799798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.800380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.800687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.800959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] PATH_INFO: `/v3/projects/afc0d5e8b6554ad0821e999888ade90b/users/0995a910405849e693e2e9b496d8a646/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.805418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc0d5e8b6554ad0821e999888ade90b, user_id=0995a910405849e693e2e9b496d8a646, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.818806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae8cc-45b0-4ed4-94a7-8b87fa7a11a4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.830555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1065/2146] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:20 2022] PUT /identity/v3/projects/afc0d5e8b6554ad0821e999888ade90b/users/0995a910405849e693e2e9b496d8a646/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:20.842129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:20.842665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:20.843084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:20.890461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] MFA Rules not processed for user `0995a910405849e693e2e9b496d8a646`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:20.891914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:20.903629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:20.906551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:20.932313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1082/2147] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:20.974927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:20.992897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd05bb-2f30-4e2a-87c2-e86edb3d2c76 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:21.066244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1066/2148] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:21.082312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:21.112294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:21.113295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:21.113591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:21.114022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] PATH_INFO: `/v3/users/59d04da51cab4d43a8d9bafe0b6560f1` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:21.117091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d04da51cab4d43a8d9bafe0b6560f1)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:21.119932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:21.187890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eee54c-c9f9-4f3b-82af-02df60b0ef82 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:21.191936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:21.192557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:21.192844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:21.193948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:21.208314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:21.220940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:21.232618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1067/2149] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:18:21.268795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:21.460308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:21.462319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:21.476081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856d4c4-3149-4f4f-84cc-1ed822807c15 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:21.480507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:21.481867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:21.481867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:21.481867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:21.484054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:21.607511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d04da51cab4d43a8d9bafe0b6560f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:21.608508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d04da51cab4d43a8d9bafe0b6560f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:21.623379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:21.625038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0dad4-6f2a-4efe-abf9-091b3730e2aa service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:21.633429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d04da51cab4d43a8d9bafe0b6560f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:21.638213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1068/2150] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:18:21.905734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d04da51cab4d43a8d9bafe0b6560f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:21.924733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-24535dbc-7654-484d-9e2b-3228964e70b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d04da51cab4d43a8d9bafe0b6560f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:21.939642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1083/2151] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:20 2022] DELETE /identity/v3/users/59d04da51cab4d43a8d9bafe0b6560f1 => generated 0 bytes in 963 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:21.941926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:21.942412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:21.942696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:22.001061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:22.004714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:22.017074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:22.017677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:22.018341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:22.114219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] MFA Rules not processed for user `660c6df77d0a4750aac30345d6768526`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:22.116404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:22.234720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1069/2152] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:22.248036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:22.249898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:22.252748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1084/2153] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:22.259251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:22.275885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:22.278090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:22.423193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c79e00-9a3a-4404-8a95-c45b59f006fa None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:22.431889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:22.432698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:22.432983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:22.433265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:22.443565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:22.445309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:22.466260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca4545-02b9-4940-94a1-d533e5c87465 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:22.468567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1085/2154] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:22.487105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:22.488218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:22.488657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:22.489124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:22.506939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:22.510265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38db65c-6c90-4b14-922e-64933e854070 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:22.526848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1070/2155] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:23.392962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:23.395401 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:23.411772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0239-b82a-4118-9ac1-8b109bca75cc service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:23.421798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:23.422754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:23.423187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:23.423639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] PATH_INFO: `/v3/projects/248f8afef2a14e12a6eafaf644d50da0` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:23.425121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=248f8afef2a14e12a6eafaf644d50da0)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:23.427248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:23.697941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248f8afef2a14e12a6eafaf644d50da0', 'request_id': 'req-bdc72668-64e9-401e-b217-d072dd293101'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:23.703896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248f8afef2a14e12a6eafaf644d50da0', 'request_id': 'req-bdc72668-64e9-401e-b217-d072dd293101'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:23.929066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248f8afef2a14e12a6eafaf644d50da0', 'request_id': 'req-bdc72668-64e9-401e-b217-d072dd293101'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:23.948567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] The token cache is being invalidate because project 248f8afef2a14e12a6eafaf644d50da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:23.948942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:23.949662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bdc72668-64e9-401e-b217-d072dd293101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:23.951409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1086/2156] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:23 2022] DELETE /identity/v3/projects/248f8afef2a14e12a6eafaf644d50da0 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:23.956550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:23.957078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:23.957258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:23.997024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:23.997024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:24.044517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1071/2157] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:24.055153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.055648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.055938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.078597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:24.079908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:24.089416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1087/2158] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:24.094383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.108326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43874f0-bb92-4bb5-af20-1fa3427d00dd None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.113893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.114660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.116034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.116524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.118583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:24.118583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.120173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:24.134598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1072/2159] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:24.138945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.151783 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa475c44-3670-4d99-a010-0eb14a048caa None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.157024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.158657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.159378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.160010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.161244 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.167075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:24.331467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1088/2160] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:24 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:24.335631 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.350514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8338dd-bbef-4c82-8428-d0eecf08ca89 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.429455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.430090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.430375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.430684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.431437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.434520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:24.660829 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1073/2161] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:24 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:24.669055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.685311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1373e-0d7d-487d-b896-33396318c59d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.698646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.699318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.699600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.699878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.700987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.709151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:24.731038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1089/2162] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:24.735447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.746879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dab48-902e-44d5-9841-53cc789a20b1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.752300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.753541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.755957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.764811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.770572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.773822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc119f-224b-4491-81a9-b9bb20441ce2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:24.796031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1074/2163] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:24.796927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:24.807652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ac104-74ca-4fb3-af75-c88984fb4731 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:24.814450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:24.815396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:24.815934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:24.816431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] PATH_INFO: `/v3/projects/1b173feb98cc4a2788019d9ad62b384f/users/0901c2e1c6114d62babc509c46d26b7d/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:24.819229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:24.839403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b173feb98cc4a2788019d9ad62b384f, user_id=0901c2e1c6114d62babc509c46d26b7d, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:24.843027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:25.055067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1090/2164] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:24 2022] PUT /identity/v3/projects/1b173feb98cc4a2788019d9ad62b384f/users/0901c2e1c6114d62babc509c46d26b7d/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:25.059152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ace8-f038-42ed-baf3-59009613f1a1 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:25.059648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ace8-f038-42ed-baf3-59009613f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:25.059728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ace8-f038-42ed-baf3-59009613f1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:25.089902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-28c2ace8-f038-42ed-baf3-59009613f1a1 None None] MFA Rules not processed for user `0901c2e1c6114d62babc509c46d26b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:25.091532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-28c2ace8-f038-42ed-baf3-59009613f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:25.230816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1075/2165] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:25.243558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:25.246842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:25.327851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc38c2-f00c-4505-a90b-400de29f90c6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:25.332325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:25.332977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:25.333400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:25.334365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:25.353964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:25.356670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ad942-25f1-4728-abef-eda308c31ac6 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:25.370379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1091/2166] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:25.996505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:27.982730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:27.983371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:27.983482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:28.011253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] MFA Rules not processed for user `660c6df77d0a4750aac30345d6768526`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:28.012508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:28.083076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1076/2167] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:28.873856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca4864-5baf-4064-b3c2-516ee717db32 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:28.874496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca4864-5baf-4064-b3c2-516ee717db32 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:28.874496 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca4864-5baf-4064-b3c2-516ee717db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:28.898595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-60ca4864-5baf-4064-b3c2-516ee717db32 None None] MFA Rules not processed for user `0901c2e1c6114d62babc509c46d26b7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:28.900074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-60ca4864-5baf-4064-b3c2-516ee717db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:28.907559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1092/2168] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:30.086431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:30.166653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:30.169418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:30.248144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b75f32-cf99-4478-96ba-08013c061a7c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:30.253972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:30.255984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:30.256374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:30.256837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] PATH_INFO: `/v3/users/60f5b19f74da48ec8466d334ab87da24` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:30.258499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f5b19f74da48ec8466d334ab87da24)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:30.260746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:30.789524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f5b19f74da48ec8466d334ab87da24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:30.790429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f5b19f74da48ec8466d334ab87da24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:30.805436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f5b19f74da48ec8466d334ab87da24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:31.147625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f5b19f74da48ec8466d334ab87da24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:31.156921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f5b19f74da48ec8466d334ab87da24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:31.166402 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1077/2169] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:30 2022] DELETE /identity/v3/users/60f5b19f74da48ec8466d334ab87da24 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:31.171273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:31.172677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:31.173133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:31.223120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:31.226432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:31.300251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1093/2170] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:31.316597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:31.318132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:31.393498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11cf4f-49cb-4d4d-bb94-a6a1245ec47e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:31.400635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:31.401339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:31.401843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:31.402321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:31.415468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:31.417062 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:31.432409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1078/2171] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:32.167763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:32.170686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:32.200221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9364a-83ff-433e-9318-da2c6a72c347 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:32.205454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:32.206807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:32.207296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:32.207599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] PATH_INFO: `/v3/projects/9cd8e6a3637a4db8b88535d0e1bd81f4` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:32.209510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd8e6a3637a4db8b88535d0e1bd81f4)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:32.212077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:32.398524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:32.406395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd8e6a3637a4db8b88535d0e1bd81f4', 'request_id': 'req-71750df6-8a2c-4a01-a4f3-578c4ff46c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:32.412284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd8e6a3637a4db8b88535d0e1bd81f4', 'request_id': 'req-71750df6-8a2c-4a01-a4f3-578c4ff46c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:32.596674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd8e6a3637a4db8b88535d0e1bd81f4', 'request_id': 'req-71750df6-8a2c-4a01-a4f3-578c4ff46c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:32.611519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] The token cache is being invalidate because project 9cd8e6a3637a4db8b88535d0e1bd81f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:32.611831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:32.612558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:32.614133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1094/2172] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:32 2022] DELETE /identity/v3/projects/9cd8e6a3637a4db8b88535d0e1bd81f4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:32.620186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:32.632613 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09a9c8-901d-4bf1-b213-7071b5955dec service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:32.690247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:32.690912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:32.691321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:32.691473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] PATH_INFO: `/v3/users/0e875c9d84cf4b26859699572d689d7f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:32.692588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e875c9d84cf4b26859699572d689d7f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:32.695192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:33.198900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e875c9d84cf4b26859699572d689d7f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:33.211692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e875c9d84cf4b26859699572d689d7f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:33.219205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e875c9d84cf4b26859699572d689d7f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:33.627499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e875c9d84cf4b26859699572d689d7f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:33.636078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e875c9d84cf4b26859699572d689d7f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:33.643279 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1079/2173] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:32 2022] DELETE /identity/v3/users/0e875c9d84cf4b26859699572d689d7f => generated 0 bytes in 1024 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:33.801801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:34.399785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:34.405906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:34.429006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750df6-8a2c-4a01-a4f3-578c4ff46c55 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:34.571958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:34.572613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:34.572924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:34.573183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] PATH_INFO: `/v3/projects/92b36f73655b4221a23e6c302f3518ad` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:34.578659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b36f73655b4221a23e6c302f3518ad)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:34.582984 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:34.875209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b36f73655b4221a23e6c302f3518ad', 'request_id': 'req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:34.887432 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b36f73655b4221a23e6c302f3518ad', 'request_id': 'req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.116294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b36f73655b4221a23e6c302f3518ad', 'request_id': 'req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.130053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] The token cache is being invalidate because project 92b36f73655b4221a23e6c302f3518ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:35.130370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.131369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.133154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1095/2174] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:34 2022] DELETE /identity/v3/projects/92b36f73655b4221a23e6c302f3518ad => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:35.138052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:35.139640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:35.161001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f5293-4f87-4269-8b7a-1e23fb50a6b1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:35.230876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:35.231597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:35.231883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:35.232156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] PATH_INFO: `/v3/users/0995a910405849e693e2e9b496d8a646` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:35.233315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0995a910405849e693e2e9b496d8a646)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:35.235425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:35.723971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0995a910405849e693e2e9b496d8a646'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.725042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0995a910405849e693e2e9b496d8a646'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.737497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:35.739006 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:35.761095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df0ee-4b6d-4e4c-8c5a-bed0061c6c83 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:35.765651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0995a910405849e693e2e9b496d8a646'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.768063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:35.769194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:35.769818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:35.770290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] PATH_INFO: `/v3/users/12b490db1ec948c69206001ed11da8a9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:35.772145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b490db1ec948c69206001ed11da8a9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:35.775103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:35.926297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0995a910405849e693e2e9b496d8a646'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.931149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0df6642d-9a9e-49b6-af70-2c7efd0fb632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0995a910405849e693e2e9b496d8a646'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:35.939093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1080/2175] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:35 2022] DELETE /identity/v3/users/0995a910405849e693e2e9b496d8a646 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:36.148292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b490db1ec948c69206001ed11da8a9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:36.149180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b490db1ec948c69206001ed11da8a9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:36.174883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b490db1ec948c69206001ed11da8a9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:36.353439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b490db1ec948c69206001ed11da8a9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:36.363454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b490db1ec948c69206001ed11da8a9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:36.371627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1096/2176] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:35 2022] DELETE /identity/v3/users/12b490db1ec948c69206001ed11da8a9 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:36.376091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:36.376600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:36.376708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:36.398948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:36.400479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:36.483622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1081/2177] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:36.495361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:36.497367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:36.579708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bed238-32b4-4d5d-b05a-601869078a4c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:36.584613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:36.589092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:36.589092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:36.589092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:36.607934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:36.609963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bd21d-01a1-4d17-b42f-fd0fde769f63 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:36.629522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1097/2178] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:36.986720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:36.989043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:37.004546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475bb37-5bc4-403c-9d74-be55d03a3517 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:37.009986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:37.010981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:37.011260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:37.011529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] PATH_INFO: `/v3/projects/afc0d5e8b6554ad0821e999888ade90b` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:37.012876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc0d5e8b6554ad0821e999888ade90b)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:37.015417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:37.300488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc0d5e8b6554ad0821e999888ade90b', 'request_id': 'req-4e3b8108-4197-4862-b4d6-476fb4d7c10c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:37.453687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc0d5e8b6554ad0821e999888ade90b', 'request_id': 'req-4e3b8108-4197-4862-b4d6-476fb4d7c10c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:37.466005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc0d5e8b6554ad0821e999888ade90b', 'request_id': 'req-4e3b8108-4197-4862-b4d6-476fb4d7c10c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:37.485222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] The token cache is being invalidate because project afc0d5e8b6554ad0821e999888ade90b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:37.485783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:37.486649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4e3b8108-4197-4862-b4d6-476fb4d7c10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:37.488693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1082/2179] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:36 2022] DELETE /identity/v3/projects/afc0d5e8b6554ad0821e999888ade90b => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:37.495502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:37.496025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:37.496256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:37.522488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:37.523864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:37.612215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1098/2180] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:37.623520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:37.624199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:37.624488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:37.648873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:37.650129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:37.659344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1083/2181] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:37.664243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:37.675835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836f6cc-c68b-49da-b07f-901959a5f3e7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:37.682951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:37.683677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:37.684028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:37.684405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:37.685264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:37.686010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:37.688294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:37.716710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1099/2182] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:37.716948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:37.728896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ef48c-7124-4e09-940a-2b459139ecb4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:37.729713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:37.761414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:37.770833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:37.770833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:37.770833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:37.772577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:37.782766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:37.881846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:37.887774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:37.944482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1084/2183] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:37 2022] POST /identity/v3/projects => generated 375 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:37.950351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:37.963651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554df31-64d1-4d7d-bb59-3729086ab581 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:38.087517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78669-f19f-4311-90a9-3607764461f9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:38.094784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:38.095106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.095692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.096172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.098036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:38.112179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:38.123067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:38.123731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.124549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.124847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] PATH_INFO: `/v3/projects/a594c7ea93d943cdaf9d8205dfec08df` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.126517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a594c7ea93d943cdaf9d8205dfec08df)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:38.128592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:38.430785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1085/2184] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:37 2022] POST /identity/v3/users => generated 288 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:38.434929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:38.451949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609625ad-33fc-4f4c-b4e6-9bd8f04396ed admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:38.477845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a594c7ea93d943cdaf9d8205dfec08df', 'request_id': 'req-524f33bc-5129-4b1d-9436-3c14c0af8bbd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:38.488431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a594c7ea93d943cdaf9d8205dfec08df', 'request_id': 'req-524f33bc-5129-4b1d-9436-3c14c0af8bbd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:38.537497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:38.538396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.538987 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.539485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.540651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:38.543675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:38.557465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1086/2185] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:38.561433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:38.571572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4cce8-19cb-4d44-95a2-9f424cda9943 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:38.576328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:38.576959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.577247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.577526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.578155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:38.579783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:38.590448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1087/2186] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:38.594747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:38.606958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100d98-9910-480b-96bc-269a02d399d0 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:38.612203 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:38.612879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.613156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.613433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] PATH_INFO: `/v3/projects/785661f334ab4fe1b20ca3c069f6d5d2/users/1fc34f594f8948caa74c6ca675077e72/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.629499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=785661f334ab4fe1b20ca3c069f6d5d2, user_id=1fc34f594f8948caa74c6ca675077e72, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:38.632778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b226878f-d3fe-423c-9045-c5bba27202fb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:38.736118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a594c7ea93d943cdaf9d8205dfec08df', 'request_id': 'req-524f33bc-5129-4b1d-9436-3c14c0af8bbd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:38.754065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] The token cache is being invalidate because project a594c7ea93d943cdaf9d8205dfec08df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:38.754364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:38.755593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-524f33bc-5129-4b1d-9436-3c14c0af8bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:38.759037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1100/2187] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:37 2022] DELETE /identity/v3/projects/a594c7ea93d943cdaf9d8205dfec08df => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:38.763649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c760-e714-402a-a98a-8cde67a03aa3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.763942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c760-e714-402a-a98a-8cde67a03aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.764303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c760-e714-402a-a98a-8cde67a03aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.807753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3a16c760-e714-402a-a98a-8cde67a03aa3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:38.809056 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3a16c760-e714-402a-a98a-8cde67a03aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:38.844896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1088/2188] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:38 2022] PUT /identity/v3/projects/785661f334ab4fe1b20ca3c069f6d5d2/users/1fc34f594f8948caa74c6ca675077e72/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:38.858668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1101/2189] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:38.867257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.869467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.869467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.870093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:38.873799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:38.874099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:38.935756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:38.937089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:38.950019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] MFA Rules not processed for user `1fc34f594f8948caa74c6ca675077e72`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:38.950909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:39.047574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1102/2190] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:39.048494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.072403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a85d7e-4ba8-49d4-8b52-8ed615ff60cc None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.080466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.080913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.081207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.081542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.082228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:39.082537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.084689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.091497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1089/2191] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:39.092458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.094849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:39.101383 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1103/2192] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:39.108235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.113760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e9f21-8510-405f-9320-e47f2fbe027b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.121206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.121836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.122114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.122389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.123022 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f0002-7925-4f61-b4a2-a3ac42303e3b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.123264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.125428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.128636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.140571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.140885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.141291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.142030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.145592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.185441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:39.343321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1090/2193] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:39 2022] POST /identity/v3/projects => generated 375 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:39.348036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.415771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1104/2194] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:39 2022] POST /identity/v3/projects => generated 369 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:39.417487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.476251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfe447-937b-4339-9fad-eb39553facb4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.513273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc40bdc-3732-4111-b2bf-52abb3e11fc7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.518807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.519767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.520240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.520852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.522744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.529421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.589134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.589973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.590129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.590415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.591215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.595315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.714907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1105/2195] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:39 2022] POST /identity/v3/users => generated 285 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:39.714907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.722491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e027f-b575-4818-94a4-d5e04c30ceec admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.728900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.729565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.729849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.730124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.730832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.732931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.746698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1106/2196] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:39.750317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.753778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1091/2197] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:39 2022] POST /identity/v3/users => generated 288 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:39.757855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.762972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bae8e-490a-45bf-b292-44ca0ca9e042 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.768736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.769644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.769976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.770241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.771009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.773576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.775603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ccf83-efb0-4901-99dc-f7ba9a422050 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.781978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.782645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.782932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.783212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.783881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.785587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.789468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1107/2198] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:39.794552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.807280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c996ff-cb93-481f-b452-2b7f4185281a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.808747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1092/2199] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:39.813061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.813389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.814182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.814563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.814987 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] PATH_INFO: `/v3/projects/37ad375058384bd8b6d2e94711e733d0/users/822502e6b8d849bfb5eeffeed87bc0f1/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.824875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb66fc-4e3b-4d94-8b5c-0aed7274e154 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.830909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.831897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.832019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.832273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.833148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.834948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.839895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ad375058384bd8b6d2e94711e733d0, user_id=822502e6b8d849bfb5eeffeed87bc0f1, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.846235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24145a2b-f32e-45ed-a7a6-367e38ac11e3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:39.855150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1093/2200] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:18:39.856881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:39.872759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7716-0ff4-4ed8-9875-48718261417b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:39.876538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:39.877579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:39.878428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:39.879262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] PATH_INFO: `/v3/projects/9064c20b6cbb49e28c27d4a1538d34f0/users/ea751bf623504c9d9cd41f0550bdc431/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:39.905325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9064c20b6cbb49e28c27d4a1538d34f0, user_id=ea751bf623504c9d9cd41f0550bdc431, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:39.909775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79229c-6c0c-48fa-aa97-d560354c8bcf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:40.024171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1108/2201] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:39 2022] PUT /identity/v3/projects/37ad375058384bd8b6d2e94711e733d0/users/822502e6b8d849bfb5eeffeed87bc0f1/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:40.025436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:40.025665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:40.026025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:40.059331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1094/2202] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:39 2022] PUT /identity/v3/projects/9064c20b6cbb49e28c27d4a1538d34f0/users/ea751bf623504c9d9cd41f0550bdc431/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:40.075129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:40.075453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:40.075765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:40.082066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] MFA Rules not processed for user `822502e6b8d849bfb5eeffeed87bc0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:40.085262 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:40.150190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] MFA Rules not processed for user `ea751bf623504c9d9cd41f0550bdc431`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:40.152738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:40.233497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1109/2203] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:40.247033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:40.248770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:40.345028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df16be-38f4-4682-9a95-f31ad6a08179 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:40.372558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:40.378736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:40.379032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:40.379311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:40.392883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:40.394660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:40.420269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1095/2204] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:40.421988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1110/2205] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:40.426376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:40.537874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37713b30-e501-4ff2-9f94-e8d379c8a035 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:40.543110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:40.543730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:40.544008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:40.544534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:40.545206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:40.547145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:40.797835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1096/2206] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:40 2022] POST /identity/v3/projects => generated 377 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:40.807416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:40.820499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510aafb-36f2-4748-8a32-1315b55d057a service glance] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:40.949128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:40.949768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:40.950108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:40.950365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:40.951088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:40.953116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.113476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1111/2207] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:40 2022] POST /identity/v3/users => generated 295 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:18:41.118735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:41.134541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a76cb-d405-4ea5-9fbf-cd99ef373d5a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:41.140518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:41.141213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.141568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.142238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.142633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:41.144546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.156363 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1097/2208] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:41.167143 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:41.181298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e429ebc-3d64-4670-bd79-da7e567889fe admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:41.188700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:41.190040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.190479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.190936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] PATH_INFO: `/v3/projects/977789170aad4374ab0760a4a4a34802/users/357acf68c611457d875e0825e8d67c92/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.220466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977789170aad4374ab0760a4a4a34802, user_id=357acf68c611457d875e0825e8d67c92, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:41.223615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.347049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1112/2209] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:41 2022] PUT /identity/v3/projects/977789170aad4374ab0760a4a4a34802/users/357acf68c611457d875e0825e8d67c92/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:41.350923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:41.362790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9119b7-0238-4356-bcc5-bbc49ed79927 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:41.432074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:41.432905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.433078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.433325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.433974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:41.435672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a8b64-8076-4764-9e33-b47065c6c505 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.448273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1098/2210] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:41.453358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:41.465894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b580ef-ead8-45f9-bb69-70ef927db3c0 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:41.471069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:41.471744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.472027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.472302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] PATH_INFO: `/v3/projects/977789170aad4374ab0760a4a4a34802/users/357acf68c611457d875e0825e8d67c92/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.474404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977789170aad4374ab0760a4a4a34802, user_id=357acf68c611457d875e0825e8d67c92, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:41.477204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.499627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1113/2211] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:41 2022] PUT /identity/v3/projects/977789170aad4374ab0760a4a4a34802/users/357acf68c611457d875e0825e8d67c92/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:41.511626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683e48e-5937-4568-95dc-3eb4e45c595c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.511913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683e48e-5937-4568-95dc-3eb4e45c595c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.512185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683e48e-5937-4568-95dc-3eb4e45c595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.542927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c683e48e-5937-4568-95dc-3eb4e45c595c None None] MFA Rules not processed for user `357acf68c611457d875e0825e8d67c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:41.544380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c683e48e-5937-4568-95dc-3eb4e45c595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:41.638078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1099/2212] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:41.661128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:41.664416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:41.756823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726ce3-b0cf-4cdf-88b9-10331c532074 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:41.762089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:41.762892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:41.763250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:41.763596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:41.848129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:41.849971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258fe4-80a5-4a09-8408-00a44543e192 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:41.888793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1114/2213] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:45.869561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:45.871122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:45.871601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:45.875241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1100/2214] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:18:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:18:45.914725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:45.921420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:45.922173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:45.947758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] MFA Rules not processed for user `48e76f49c2224e959267952f79669906`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:45.949167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:46.126914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1115/2215] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:18:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:46.150144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:46.152857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:46.194945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:46.197597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:46.198757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950051-a353-49df-83fd-b5b37b799ae4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:46.218986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:46.221683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:46.222192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:46.223664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:46.242424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9d722-0a6e-4c88-a51e-ff3394ce35c0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:46.251085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:46.251721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:46.254225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:46.255822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:46.256786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:46.258416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:46.288871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1101/2216] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:46.320517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:46.322335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:46.398385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:46.400131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:46.418417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1116/2217] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:46.491862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951611f7-e0a4-4dc2-9d33-b8cc29480c29 service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:46.498388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:46.499341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:46.499863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:46.500333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] PATH_INFO: `/v3/users/e69372ff650d4a94a8b9fa56d6a62d0f` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:46.502347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69372ff650d4a94a8b9fa56d6a62d0f)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:46.505512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:47.046365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69372ff650d4a94a8b9fa56d6a62d0f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:47.047832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69372ff650d4a94a8b9fa56d6a62d0f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:47.063699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69372ff650d4a94a8b9fa56d6a62d0f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:47.111088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:47.213520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69372ff650d4a94a8b9fa56d6a62d0f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:47.223247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69372ff650d4a94a8b9fa56d6a62d0f'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:47.234224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1102/2218] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:46 2022] DELETE /identity/v3/users/e69372ff650d4a94a8b9fa56d6a62d0f => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:47.657039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:47.702562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0dc95-9416-41d2-9fe5-2a027e6f39b9 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:47.784616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:47.785295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:47.785598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:47.785943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] PATH_INFO: `/v3/projects/68ed45c4bf4f4ef383577dfbd506805e` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:47.787261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ed45c4bf4f4ef383577dfbd506805e)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:47.789336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:48.022942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ed45c4bf4f4ef383577dfbd506805e', 'request_id': 'req-7fde759e-d1f8-4535-86c5-dd00b9b276c1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:48.031219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ed45c4bf4f4ef383577dfbd506805e', 'request_id': 'req-7fde759e-d1f8-4535-86c5-dd00b9b276c1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:48.171566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ed45c4bf4f4ef383577dfbd506805e', 'request_id': 'req-7fde759e-d1f8-4535-86c5-dd00b9b276c1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:48.186992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] The token cache is being invalidate because project 68ed45c4bf4f4ef383577dfbd506805e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:48.187288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:48.188183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:48.190227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1117/2219] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:47 2022] DELETE /identity/v3/projects/68ed45c4bf4f4ef383577dfbd506805e => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:48.194888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:48.196421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:48.213566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd672252-151d-4ad3-87ce-4281dbae1a7d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:48.337712 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:48.338529 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:48.338882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:48.339205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] PATH_INFO: `/v3/users/0c914c46614a4495ab06e7af013f683c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:48.340628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c914c46614a4495ab06e7af013f683c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:48.342917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:49.115761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c914c46614a4495ab06e7af013f683c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:49.116974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c914c46614a4495ab06e7af013f683c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:49.154541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c914c46614a4495ab06e7af013f683c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:49.384332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c914c46614a4495ab06e7af013f683c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:49.393321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c914c46614a4495ab06e7af013f683c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:49.400422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1103/2220] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:48 2022] DELETE /identity/v3/users/0c914c46614a4495ab06e7af013f683c => generated 0 bytes in 1206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:49.685133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:49.689901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:49.783878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde759e-d1f8-4535-86c5-dd00b9b276c1 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:49.857527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:49.858277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:49.858569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:49.858930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:49.864330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:49.881492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:49.883159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:49.970933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:49.972476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2c2bf-5ead-4e3c-a756-a3e808034165 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:49.982565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facde0fc-63d7-4dab-b238-329786326d98 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:49.993898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:49.993898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:49.993898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:49.993898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] PATH_INFO: `/v3/projects/8d9d9069e3e543a98ef75b2461cf3242` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:49.994875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1118/2221] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:49.994984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9d9069e3e543a98ef75b2461cf3242)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:49.994984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:50.122073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:50.216367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9d9069e3e543a98ef75b2461cf3242', 'request_id': 'req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:50.453751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9d9069e3e543a98ef75b2461cf3242', 'request_id': 'req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:50.466328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9d9069e3e543a98ef75b2461cf3242', 'request_id': 'req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:50.472076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947342-7ff6-4bb4-bd28-2a5dab598783 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:50.473119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947342-7ff6-4bb4-bd28-2a5dab598783 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:50.473424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947342-7ff6-4bb4-bd28-2a5dab598783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:50.483054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] The token cache is being invalidate because project 8d9d9069e3e543a98ef75b2461cf3242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:50.483347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:50.484016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b9c1d72b-b1a4-4f07-8226-4a7da3944e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:50.485865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1104/2222] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:49 2022] DELETE /identity/v3/projects/8d9d9069e3e543a98ef75b2461cf3242 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:50.495326 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:50.495738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:50.496029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:50.503513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-fd947342-7ff6-4bb4-bd28-2a5dab598783 None None] MFA Rules not processed for user `8d154e96376a49918658b0c340c722e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:50.507449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fd947342-7ff6-4bb4-bd28-2a5dab598783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:50.531146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:50.532717 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:50.601275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1119/2223] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:50.637068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1105/2224] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:18:50.654170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:50.655052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:50.657357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:50.687752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:50.688884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:50.698557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1120/2225] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:50.702941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:50.714161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fff8fa-6d5e-40bb-b3e1-7644381c2861 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:50.727711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:50.728388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:50.728670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:50.728960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:50.729668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:18:50.730015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:50.731802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:50.749899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1106/2226] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:18:50.755085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:50.769296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4785169-63ff-42c5-ac46-9c40283c3c0d None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:50.781439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:50.782410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:50.782885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:50.783316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:50.784258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:50.787534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:50.991303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1121/2227] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:18:50 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:50.996857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:51.019910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2e247-dba5-4f35-9359-8dacad3cb6e3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:51.116822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:51.117665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:51.118018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:51.118368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:51.119238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:51.122715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:51.325709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1107/2228] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:18:50 2022] POST /identity/v3/users => generated 292 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:18:51.330354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:51.343669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2f871-f8b2-4a03-88df-69fb870978bd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:51.349955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:51.350606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:51.350949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:51.351232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:51.351909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:51.355586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:51.369470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1122/2229] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:51.385088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:51.398208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89da8b-b498-4b95-abf8-1c5990e2288d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:51.404330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:51.404977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:51.405248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:51.405570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:51.406292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:51.408059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ceb8d-ae7d-460c-ac1a-a70a3ca3b969 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:51.420092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1108/2230] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:18:51.425294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:51.436821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b74d-3bea-4ca6-90f6-37206de43f2b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:51.443160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:51.444489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:51.444691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:51.445170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] PATH_INFO: `/v3/projects/9b485557d9684a43885c09cff478d91d/users/82b379a985694bf899bcb5e9aaa76bfe/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:51.474936 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b485557d9684a43885c09cff478d91d, user_id=82b379a985694bf899bcb5e9aaa76bfe, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:51.484795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:51.821186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1123/2231] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:18:51 2022] PUT /identity/v3/projects/9b485557d9684a43885c09cff478d91d/users/82b379a985694bf899bcb5e9aaa76bfe/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:51.833249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:51.834021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:51.834302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:51.883012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] MFA Rules not processed for user `82b379a985694bf899bcb5e9aaa76bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:51.884386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:51.960609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1109/2232] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:51.975174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:51.977157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:52.040567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d227862-386d-4d0d-b824-a332d2c96cc9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:52.045012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:52.045713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:52.046010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:52.046287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:52.056453 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:52.057917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff86515-fe33-4dcb-852f-b6ad9728bd65 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:52.069431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1124/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:18:57.142106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:57.143638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:57.220693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b268a49-68c1-4376-a8ea-24ee6173ec3d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:57.226547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:57.227207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:57.227484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:57.227758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] PATH_INFO: `/v3/users/1fc34f594f8948caa74c6ca675077e72` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:57.228881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc34f594f8948caa74c6ca675077e72)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:57.230870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:57.570188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc34f594f8948caa74c6ca675077e72'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:57.571010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc34f594f8948caa74c6ca675077e72'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:57.584909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc34f594f8948caa74c6ca675077e72'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:57.724385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc34f594f8948caa74c6ca675077e72'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:57.732668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc34f594f8948caa74c6ca675077e72'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:57.738672 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1110/2234] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:57 2022] DELETE /identity/v3/users/1fc34f594f8948caa74c6ca675077e72 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:18:57.743683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:57.744199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:57.744294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:57.766194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:18:57.767583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:57.830348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1125/2235] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:18:57.841703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:57.843471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:57.911456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a5250-e43b-4f55-86f6-c72c496332c4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:57.915676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:57.916300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:57.916587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:57.916866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:57.927200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:57.936488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:57.939098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1111/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:18:58.747551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:58.749129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:18:58.762024 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ab82e-f7a3-4ac8-bb04-12f5dab1c216 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:58.769966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:58.771108 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:58.771568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:58.772135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] PATH_INFO: `/v3/projects/785661f334ab4fe1b20ca3c069f6d5d2` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:58.773511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785661f334ab4fe1b20ca3c069f6d5d2)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:58.776266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:58.956774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785661f334ab4fe1b20ca3c069f6d5d2', 'request_id': 'req-c753b32e-33e9-49df-bee2-95c214e6cbe6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:58.965462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785661f334ab4fe1b20ca3c069f6d5d2', 'request_id': 'req-c753b32e-33e9-49df-bee2-95c214e6cbe6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.014112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:18:59.127216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785661f334ab4fe1b20ca3c069f6d5d2', 'request_id': 'req-c753b32e-33e9-49df-bee2-95c214e6cbe6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.139818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] The token cache is being invalidate because project 785661f334ab4fe1b20ca3c069f6d5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:18:59.140105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.140833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.142438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1126/2237] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:18:58 2022] DELETE /identity/v3/projects/785661f334ab4fe1b20ca3c069f6d5d2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:18:59.147589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:18:59.163153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b960-b8e6-49fb-8de0-2d778648d3f0 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:18:59.223330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:18:59.224591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:18:59.224865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:18:59.225131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] PATH_INFO: `/v3/users/ea751bf623504c9d9cd41f0550bdc431` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:18:59.226317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea751bf623504c9d9cd41f0550bdc431)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:18:59.228365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:18:59.713059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea751bf623504c9d9cd41f0550bdc431'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.714089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea751bf623504c9d9cd41f0550bdc431'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.732141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea751bf623504c9d9cd41f0550bdc431'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.846922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea751bf623504c9d9cd41f0550bdc431'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.857949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea751bf623504c9d9cd41f0550bdc431'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:18:59.866683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1112/2238] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:18:59 2022] DELETE /identity/v3/users/ea751bf623504c9d9cd41f0550bdc431 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:00.610463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:00.612575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:00.629880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753b32e-33e9-49df-bee2-95c214e6cbe6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:00.706093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:00.707096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:00.707420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:00.707707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] PATH_INFO: `/v3/projects/9064c20b6cbb49e28c27d4a1538d34f0` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:00.709014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9064c20b6cbb49e28c27d4a1538d34f0)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:00.711556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:00.786042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:00.884965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9064c20b6cbb49e28c27d4a1538d34f0', 'request_id': 'req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:00.893038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9064c20b6cbb49e28c27d4a1538d34f0', 'request_id': 'req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.004249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9064c20b6cbb49e28c27d4a1538d34f0', 'request_id': 'req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.018646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] The token cache is being invalidate because project 9064c20b6cbb49e28c27d4a1538d34f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:01.018955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.019636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.025543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1127/2239] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:00 2022] DELETE /identity/v3/projects/9064c20b6cbb49e28c27d4a1538d34f0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:01.026886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:01.028918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:01.043593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a72e5e-6d42-453e-be23-bb6c1c796670 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:01.122280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:01.123639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:01.124130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:01.124656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] PATH_INFO: `/v3/users/357acf68c611457d875e0825e8d67c92` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:01.127454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=357acf68c611457d875e0825e8d67c92)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:01.131441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:01.673675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357acf68c611457d875e0825e8d67c92'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.674554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357acf68c611457d875e0825e8d67c92'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.688795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357acf68c611457d875e0825e8d67c92'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.875954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357acf68c611457d875e0825e8d67c92'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.891362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-33501085-3ec1-4a50-a50a-060e2b1f29bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357acf68c611457d875e0825e8d67c92'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:01.902290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1113/2240] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:19:01 2022] DELETE /identity/v3/users/357acf68c611457d875e0825e8d67c92 => generated 0 bytes in 876 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:02.645068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:02.646855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:02.663550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3ffc2-795b-4b09-9b4c-7bfa540d5fc6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:02.734635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:02.735318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:02.735606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:02.735890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] PATH_INFO: `/v3/projects/977789170aad4374ab0760a4a4a34802` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:02.737063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] RBAC: Authorizing `identity:delete_project(project_id=977789170aad4374ab0760a4a4a34802)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:02.739044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:02.994358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977789170aad4374ab0760a4a4a34802', 'request_id': 'req-cb305a56-42db-4a18-ba69-8308320a312e'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:03.000935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977789170aad4374ab0760a4a4a34802', 'request_id': 'req-cb305a56-42db-4a18-ba69-8308320a312e'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:03.176043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977789170aad4374ab0760a4a4a34802', 'request_id': 'req-cb305a56-42db-4a18-ba69-8308320a312e'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:03.190137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] The token cache is being invalidate because project 977789170aad4374ab0760a4a4a34802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:03.190439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:03.191148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-cb305a56-42db-4a18-ba69-8308320a312e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:03.193089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1128/2241] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:02 2022] DELETE /identity/v3/projects/977789170aad4374ab0760a4a4a34802 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:03.200825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:03.201369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:03.201501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:03.225859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:03.227121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:03.306333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1114/2242] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:03.314468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:03.314793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:03.315075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:03.338470 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:03.341201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:03.358149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1129/2243] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:03.358414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:03.369824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1e3b8-63d6-4aee-a8a2-91a90b8ddbd3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:03.375445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:03.376067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:03.376344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:03.376616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:03.377274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:19:03.377575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:03.379374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:03.395026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1115/2244] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:19:03.400568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:03.412045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48158ed3-c1db-4c22-850b-bfbae2f27104 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:03.418214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:03.418895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:03.419179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:03.419457 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:03.420364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:03.423223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:03.764661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1130/2245] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:03 2022] POST /identity/v3/projects => generated 385 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:03.771902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:03.790557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b290b0-2c39-4b41-8ee2-91587905c3df admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:03.862774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:03.863608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:03.864098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:03.864629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:03.866302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:03.869605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:04.038969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1116/2246] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:03 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:19:04.040200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:04.052651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7742a-e198-4fc1-bbe3-551df8d8a2ce admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:04.057613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:04.058315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:04.058699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:04.059044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:04.059791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:04.061571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:04.075747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1131/2247] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:04.080011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:04.104056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f3652-7d0e-4953-84c3-698291367516 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:04.111253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:04.112338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:04.112611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:04.113209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:04.113994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:04.116989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac3a12-9f5f-46eb-9196-874ad1700e3d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:04.131234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1117/2248] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:19:04.135809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:04.149057 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f158d-9ea8-4986-a70e-fe80dd043399 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:04.154494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:04.155209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:04.155497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:04.155779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] PATH_INFO: `/v3/projects/01817da1260c41b4abe6f02b11c66232/users/61318fafe2a1469c884b5290fe4ba366/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:04.175212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01817da1260c41b4abe6f02b11c66232, user_id=61318fafe2a1469c884b5290fe4ba366, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:04.179172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:04.370182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1132/2249] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:04 2022] PUT /identity/v3/projects/01817da1260c41b4abe6f02b11c66232/users/61318fafe2a1469c884b5290fe4ba366/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:04.375628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6c11-f36f-4ecb-8c0d-d58c6d3da218 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:04.376145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6c11-f36f-4ecb-8c0d-d58c6d3da218 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:04.376239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e6c11-f36f-4ecb-8c0d-d58c6d3da218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:04.398245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-cb9e6c11-f36f-4ecb-8c0d-d58c6d3da218 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:04.399514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-cb9e6c11-f36f-4ecb-8c0d-d58c6d3da218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:04.466239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1118/2250] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:04.476975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:04.479007 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:04.545031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00624b-f16b-4bef-884a-9d455e238401 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:04.548846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:04.549458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:04.549735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:04.550011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:04.563475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:04.564990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:04.575033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1133/2251] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:19:05.258997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:05.606917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:05.607763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:05.607763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:05.632044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:05.633937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:05.712486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1119/2252] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:19:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:05.766834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:05.768360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:05.879553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04363d3-c78b-41da-bd7e-7ed7a36f7390 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:05.888973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:05.890008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:05.890555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:05.891410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:05.914611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:05.916384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855966b7-69bc-4db3-bf0e-13f3e52adc6c service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:05.927484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1134/2253] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:19:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:19:06.894910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:06.896648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:06.945678 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a62a0-f7c5-4515-ba3b-df05a61267f6 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:06.959520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:06.959520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:06.959520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:06.959520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] PATH_INFO: `/v3/users/82b379a985694bf899bcb5e9aaa76bfe` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:06.960995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b379a985694bf899bcb5e9aaa76bfe)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:06.961538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:07.000123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:07.167132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:07.167610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:07.167699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:07.195354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:07.196738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:07.207158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1135/2254] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:19:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:07.231463 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:07.253300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f5f0e-83ac-411f-b87f-c2dd8e831f35 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:07.260146 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:07.261090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:07.261947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:07.262814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:07.276743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:07.278719 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba13c3-85e2-445a-8c17-0829a41491d1 service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:07.525756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1136/2255] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:19:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:19:07.633825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b379a985694bf899bcb5e9aaa76bfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:07.634858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b379a985694bf899bcb5e9aaa76bfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:07.651614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b379a985694bf899bcb5e9aaa76bfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:08.079661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b379a985694bf899bcb5e9aaa76bfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:08.095282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b379a985694bf899bcb5e9aaa76bfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:08.106310 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1120/2256] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:19:06 2022] DELETE /identity/v3/users/82b379a985694bf899bcb5e9aaa76bfe => generated 0 bytes in 1215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:08.112060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:08.112448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:08.112900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:08.139982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:08.141939 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:08.237872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1137/2257] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:08.249856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:08.285049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:08.324662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1170b5a-d692-4e31-9ae1-d5a2bb9c480a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:08.331054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:08.331054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:08.331054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:08.331418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:08.346401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:08.350723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7c0d7-4901-4823-a440-5009b1dc5d88 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:08.371764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1121/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:09.255667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:09.255667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:09.272664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5b8c3-94aa-44d5-bdfc-56191465e185 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:09.278478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:09.279304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:09.279597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:09.279881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] PATH_INFO: `/v3/projects/9b485557d9684a43885c09cff478d91d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:09.292280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b485557d9684a43885c09cff478d91d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:09.296759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:09.460962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:09.461466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:09.461561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:09.479994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b485557d9684a43885c09cff478d91d', 'request_id': 'req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:09.485513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:09.487752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b485557d9684a43885c09cff478d91d', 'request_id': 'req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:09.487963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:09.561889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1122/2259] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:19:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:09.576125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:09.578388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:09.682979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeebe758-58c8-49cb-a383-d627fda2f97c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:09.688088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b485557d9684a43885c09cff478d91d', 'request_id': 'req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:09.689816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:09.690378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:09.690692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:09.692095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:09.701982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:09.703749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a984732-8dd1-4b2e-8c45-5e4e83b9f346 service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:09.708830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] The token cache is being invalidate because project 9b485557d9684a43885c09cff478d91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:09.709307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:09.710685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5d47941e-d7a3-4dc8-96b7-9b9bd97fbbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:09.712672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1138/2260] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:09 2022] DELETE /identity/v3/projects/9b485557d9684a43885c09cff478d91d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:09.715936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1123/2261] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:19:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 06:19:09.719624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:09.720048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:09.720566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:09.745789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:09.747151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:09.848195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1139/2262] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:09.858884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:09.859496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:09.860165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:09.954966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:09.960420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:10.026413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1124/2263] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:10.029910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:10.048386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a045d-08c4-4ac3-b94d-3fe937b616f8 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:10.062962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:10.063757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:10.064147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:10.064540 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:10.065427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:19:10.066415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:10.069340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:10.100105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1140/2264] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:19:10.101108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:10.153417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca8486-2f6d-415a-a849-ba101bce49ea None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:10.186084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:10.186761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:10.187047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:10.187319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:10.187972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:10.203515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd645-0d34-4251-81e0-907fe729ceb3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:10.608298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:10.610170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1125/2265] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:10 2022] POST /identity/v3/projects => generated 359 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:19:10.621297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9879ee-de8f-4dbd-8827-227814966484 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:10.759474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:10.760113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:10.760392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:10.760671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:10.761314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:10.768073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:10.980470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:10.981347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:10.981876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.020207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:11.023775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:11.120029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1126/2266] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:19:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:11.143231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:11.145221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:11.218261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1141/2267] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:10 2022] POST /identity/v3/users => generated 280 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:11.224557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:11.256801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023b4ef-b3d5-4607-9f48-3fcd4f435744 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:11.280941 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:11.282541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:11.283238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:11.283756 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.289090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:11.294275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:11.307131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb5415-48ba-4256-982b-4803e6db32df None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:11.322176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1142/2268] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:19:11.326207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:11.331404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:11.333750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:11.335976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:11.336681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.340101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3165e3-5aff-4a25-8a1a-96570749415d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:11.366220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:11.366961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:11.368689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:11.369020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.369697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:11.378908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c83c4-5f07-46f3-82a9-3da5fa6326e0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:11.383242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:11.385420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:11.408936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1127/2269] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:19:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:19:11.414717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1143/2270] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:19:11.419072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:11.443846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94356573-a9e7-467a-acd3-82cc49fe1685 service swift] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:11.452866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:11.453969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:11.454394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:11.454872 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] PATH_INFO: `/v3/projects/e0ae00ca7d284c14b4ba6e01f2e8c470/users/25831a8bead347db9053e90b3b563e09/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.487023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ae00ca7d284c14b4ba6e01f2e8c470, user_id=25831a8bead347db9053e90b3b563e09, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:11.490757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:11.712347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1128/2271] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:11 2022] PUT /identity/v3/projects/e0ae00ca7d284c14b4ba6e01f2e8c470/users/25831a8bead347db9053e90b3b563e09/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:11.733290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:11.733290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:11.734358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:11.811420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] MFA Rules not processed for user `25831a8bead347db9053e90b3b563e09`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:11.813911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:11.973174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1144/2272] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:12.039053 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:12.042909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:12.164661 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e1cfe-8790-4565-b1df-326a13431793 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:12.171547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:12.172578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:12.173064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:12.173822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:12.186858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:12.190401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fabea3-4fa5-4c21-abcb-81cac95f5068 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:12.211119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1129/2273] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:12.231065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:12.359533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69323fcc-0f9e-46a9-91c8-08f2d60070be None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:12.370990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:12.372175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:12.372685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:12.373195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] PATH_INFO: `/v3/users/25831a8bead347db9053e90b3b563e09` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:12.375168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25831a8bead347db9053e90b3b563e09)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:12.378883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:13.465644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25831a8bead347db9053e90b3b563e09'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:13.471272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25831a8bead347db9053e90b3b563e09'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:13.499662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25831a8bead347db9053e90b3b563e09'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:13.622318 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25831a8bead347db9053e90b3b563e09'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:13.637255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25831a8bead347db9053e90b3b563e09'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:13.646988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1145/2274] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:19:12 2022] DELETE /identity/v3/users/25831a8bead347db9053e90b3b563e09 => generated 0 bytes in 1417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:13.652512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:13.653282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:13.653282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:13.678691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:13.679976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:13.763753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1130/2275] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:13.775634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:13.777281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:13.911709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfb0b1-0835-4ca3-8bd0-60399891bcb2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:13.920422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:13.923145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:13.923654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:13.924697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:13.944378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:13.945847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0aa61-1e0e-44e7-9bd0-46581745585e service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:13.964929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1146/2276] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:14.402807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:15.622440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:15.623789 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:15.648355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ce179-e99c-423f-96f3-400972a5b6a0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:15.659324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:15.659990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:15.662270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:15.662584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] PATH_INFO: `/v3/projects/e0ae00ca7d284c14b4ba6e01f2e8c470` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:15.667382 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ae00ca7d284c14b4ba6e01f2e8c470)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:15.669638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:15.691651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:15.984930 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ae00ca7d284c14b4ba6e01f2e8c470', 'request_id': 'req-8a411c79-cce6-4cb0-8484-437e125d51d8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:16.200785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ae00ca7d284c14b4ba6e01f2e8c470', 'request_id': 'req-8a411c79-cce6-4cb0-8484-437e125d51d8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:16.219406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ae00ca7d284c14b4ba6e01f2e8c470', 'request_id': 'req-8a411c79-cce6-4cb0-8484-437e125d51d8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:16.255244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] The token cache is being invalidate because project e0ae00ca7d284c14b4ba6e01f2e8c470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:16.255966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:16.257556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a411c79-cce6-4cb0-8484-437e125d51d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:16.262200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1131/2277] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:15 2022] DELETE /identity/v3/projects/e0ae00ca7d284c14b4ba6e01f2e8c470 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:16.268840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:16.271562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:16.272407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:16.351308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:16.354014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:16.587764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1147/2278] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:16.590435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:16.590879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:16.591324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:16.643735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:16.646647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:16.685544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1132/2279] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:16.693105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:16.724616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f1ce0-0ea7-4abe-bd39-e7132a985b10 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:16.742578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:16.743910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:16.744454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:16.744964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:16.748614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:19:16.749011 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:16.750983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:16.769963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1148/2280] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:19:16.773944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:16.788144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe541f58-2b45-4d73-9928-4353ebb8f213 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:16.793563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:16.794261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:16.804709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:16.804709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:16.804709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:16.804709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8200-41af-42e7-9ddc-6a7503f9b624 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:17.118361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:17.118361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:17.118361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b2337-85c7-4c9f-9768-089bd3a12267 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:17.120581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:17.120581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.120581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.120581 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] PATH_INFO: `/v3/users/0901c2e1c6114d62babc509c46d26b7d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.121974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0901c2e1c6114d62babc509c46d26b7d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:17.121974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:17.188281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1133/2281] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:16 2022] POST /identity/v3/projects => generated 373 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:19:17.190057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.190568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.191103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.235532 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] MFA Rules not processed for user `61318fafe2a1469c884b5290fe4ba366`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:17.236770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:17.337721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1134/2282] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:17.338612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:17.435232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc50b4-e414-4972-959f-fddd1cae0967 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:17.441319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0901c2e1c6114d62babc509c46d26b7d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:17.441942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:17.442380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0901c2e1c6114d62babc509c46d26b7d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:17.443590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.443996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.445488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.446502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:17.449192 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:17.461675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0901c2e1c6114d62babc509c46d26b7d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:17.669707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0901c2e1c6114d62babc509c46d26b7d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:17.669707 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f70e872d-ba42-4bfa-8504-ae370f514f1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0901c2e1c6114d62babc509c46d26b7d'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:17.671292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1135/2284] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:17 2022] POST /identity/v3/users => generated 287 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:17.671292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:17.671833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1149/2284] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:19:16 2022] DELETE /identity/v3/users/0901c2e1c6114d62babc509c46d26b7d => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:17.671937 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:17.672029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.672029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.672029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.685892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50143-f85b-4e4a-8bdc-a1c865ab8572 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:17.694744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:17.696953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:17.769019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:17.770019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.771027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.771531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.772362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:17.774579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:17.802569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1150/2285] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:17.802569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:17.802569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:17.815227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4a9e2-9ad5-4ef3-b5e1-8c9419d7284c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:17.821246 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:17.821875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:17.822163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:17.823106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:17.823353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:17.824881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:17.847997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1151/2286] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:17.847997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:17.847997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:18.026924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51c424-1902-4a9d-b9a6-72492b4c2e73 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.032972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.033349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.033662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.034058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.046512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.048465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.061731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1152/2287] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:18.062634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.073546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04487-bba9-48d2-9940-e1292d7b0c57 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.093607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1136/2288] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:17 2022] POST /identity/v3/projects => generated 385 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:18.096202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.190643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.191354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.191642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.191976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.192902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.195134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.231142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1153/2289] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:18.236198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.255185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beef0a0-37bc-421e-99c6-5224a95cd56e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.258306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5bfe-7c56-466d-a610-49d77c014a28 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.268402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.269451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.269895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.270545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] PATH_INFO: `/v3/projects/b38f9319c7534904ac58d1d9f9048b91/users/2cffd4ac2817448e94d2b32ac7ffcede/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.271148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.272651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.273290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.273891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.275498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.280598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.323546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38f9319c7534904ac58d1d9f9048b91, user_id=2cffd4ac2817448e94d2b32ac7ffcede, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.323546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866ecf4-fbe6-4fe0-a582-2a24457e5f60 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.519908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1154/2290] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:18 2022] PUT /identity/v3/projects/b38f9319c7534904ac58d1d9f9048b91/users/2cffd4ac2817448e94d2b32ac7ffcede/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:18.521614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1137/2291] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:18 2022] POST /identity/v3/users => generated 299 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:18.528136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.529403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.531014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.531833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.555445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:18.565612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bef36c-2ba5-434e-aa54-f51453751ded admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.565767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:18.642447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.643188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.643552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.643971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.644790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.647961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.657315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1155/2292] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:18.664657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1138/2293] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:18.668091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.668588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.669524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:18.681183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6d147-8adc-477c-8168-c2a4f97e7a9e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.689803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.690487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.690838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.691128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] PATH_INFO: `/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62/users/001ca23cef854ad1b8a234d80279bcfe/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.715818 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed8b47c2b8a495bb5fdb68cd8087d62, user_id=001ca23cef854ad1b8a234d80279bcfe, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.718925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.757430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f49a0-e701-4c38-afed-2d1a54e79c06 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.761536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.762160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.762443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.763338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.774655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.776135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.786410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1156/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:18.787140 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:18.788487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:18.803606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff31e7-12b5-484a-b59f-052469d64006 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:18.808946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:18.809589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:18.810113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:18.810226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] PATH_INFO: `/v3/projects/1b173feb98cc4a2788019d9ad62b384f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:18.811843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b173feb98cc4a2788019d9ad62b384f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:18.813827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:18.934280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1139/2295] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:18 2022] PUT /identity/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62/users/001ca23cef854ad1b8a234d80279bcfe/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:18.939404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:19.057507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf309f-61c1-4f63-8d7a-4f362c722f1f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:19.073376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:19.074806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.075476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.075888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.077038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:19.081113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:19.105266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1140/2296] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:19.110865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:19.116324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b173feb98cc4a2788019d9ad62b384f', 'request_id': 'req-62668c5b-f1c3-4247-a6e4-a01e20c36a73'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:19.123275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b173feb98cc4a2788019d9ad62b384f', 'request_id': 'req-62668c5b-f1c3-4247-a6e4-a01e20c36a73'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:19.125563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558159af-1fdc-47fd-90ec-38259e65cc20 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:19.229748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:19.230971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.231715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.232209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] PATH_INFO: `/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62/users/001ca23cef854ad1b8a234d80279bcfe/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.236015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed8b47c2b8a495bb5fdb68cd8087d62, user_id=001ca23cef854ad1b8a234d80279bcfe, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:19.243406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ceb5e-8b49-49ef-85a7-3081cdcd3bf4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:19.266010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1141/2297] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:19 2022] PUT /identity/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62/users/001ca23cef854ad1b8a234d80279bcfe/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:19.298917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b173feb98cc4a2788019d9ad62b384f', 'request_id': 'req-62668c5b-f1c3-4247-a6e4-a01e20c36a73'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:19.332167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] The token cache is being invalidate because project 1b173feb98cc4a2788019d9ad62b384f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:19.332765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:19.344488 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-62668c5b-f1c3-4247-a6e4-a01e20c36a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:19.353954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1157/2298] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:18 2022] DELETE /identity/v3/projects/1b173feb98cc4a2788019d9ad62b384f => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:19.355171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.365149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.365149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.385246 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:19.387674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:19.549881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1142/2299] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:19.561019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.562239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.562653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.596261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:19.597666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:19.610575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1158/2300] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:19.615168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:19.631312 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef544d-9bf4-412c-9294-ea66e7e7b184 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:19.637270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:19.637911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.638198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.638475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.639313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:19:19.639628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:19.641283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:19.654667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1143/2301] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:19:19.658988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:19.671242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4ea6-86e8-4ef0-a6f2-3a906bbe9d06 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:19.676732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:19.677651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:19.678405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:19.678405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:19.678928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:19.680880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:20.061880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1159/2302] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:19 2022] POST /identity/v3/projects => generated 377 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:19:20.062764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:20.075278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35e2e1-10d7-4708-93df-7867e879e19e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:20.152446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:20.153198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:20.153573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:20.155032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:20.155032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:20.156969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:20.353130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1144/2303] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:20 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:20.357637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:20.374889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9c5c9-0921-40b1-87e4-3eadf222ebfe admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:20.384241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:20.385250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:20.385593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:20.386014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:20.387407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:20.390269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:20.415502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1160/2304] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:20.417274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:20.433741 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26572b2e-df34-48cd-b8f5-9b83370225f1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:20.441332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:20.442194 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:20.443207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:20.444102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:20.444985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:20.447076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6fa0e-fde6-4e7f-b806-2cdb791bd3a2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:20.462152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1145/2305] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:20.467181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:20.491342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542cace-2f86-4906-aedf-b15a31767519 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:20.498052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:20.498828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:20.499075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:20.499475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] PATH_INFO: `/v3/projects/fde0e661bd5646bab91b5e6b623a2e7e/users/2f141efe3bb74587b5ba13f0cd5d89db/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:20.539356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde0e661bd5646bab91b5e6b623a2e7e, user_id=2f141efe3bb74587b5ba13f0cd5d89db, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:20.547748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:20.764680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1161/2306] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:20 2022] PUT /identity/v3/projects/fde0e661bd5646bab91b5e6b623a2e7e/users/2f141efe3bb74587b5ba13f0cd5d89db/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:20.773007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb378cb-dffa-43c3-8eac-6b02c50c05a7 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:20.774492 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb378cb-dffa-43c3-8eac-6b02c50c05a7 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:20.774837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb378cb-dffa-43c3-8eac-6b02c50c05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:20.817147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5fb378cb-dffa-43c3-8eac-6b02c50c05a7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:20.821339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5fb378cb-dffa-43c3-8eac-6b02c50c05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:20.908710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1146/2307] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:20.924041 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:20.929519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:21.020641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e90a8d-9b3f-4c79-b45e-cd16a2efa674 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:21.031812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:21.033070 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:21.033564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:21.034021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:21.050802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:21.053160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:21.070860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1162/2308] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:22.154567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b873-e40c-4bcc-8add-c5090c84e777 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:22.155992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b873-e40c-4bcc-8add-c5090c84e777 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:22.156747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244b873-e40c-4bcc-8add-c5090c84e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:22.252496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0244b873-e40c-4bcc-8add-c5090c84e777 None None] MFA Rules not processed for user `2cffd4ac2817448e94d2b32ac7ffcede`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:22.253985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0244b873-e40c-4bcc-8add-c5090c84e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:22.372204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1147/2309] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:22.392586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:22.396096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:22.499191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe204e61-c0fb-4487-a896-447ee39ba507 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:22.505297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:22.506366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:22.506979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:22.507374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:22.518586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:22.520590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e079b02-e7e9-4670-baca-75b85ba9df23 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:22.534187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1163/2310] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:24.497888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:25.583589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:25.669132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:25.669675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:25.670095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:25.725116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] MFA Rules not processed for user `2cffd4ac2817448e94d2b32ac7ffcede`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:25.731704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:25.762397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1148/2311] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:30.933063 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:30.934104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:30.934571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:31.022324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] MFA Rules not processed for user `001ca23cef854ad1b8a234d80279bcfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:31.024712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:31.195424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1164/2312] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:31.200746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:31.202826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:31.281088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0b4d-ead1-4fa4-bed0-8ec7df07fecd None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:31.286263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:31.287410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:31.287843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:31.288371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:31.291850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:31.368256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:31.369960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2760342-d100-404a-9d5d-24cc6181ff1a service glance] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:31.401108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1149/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:31.623934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:31.625507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:31.641423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2490e-7740-4a4f-a19c-0b5c8a551232 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:31.651124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:31.651843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:31.652128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:31.652405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] PATH_INFO: `/v3/users/822502e6b8d849bfb5eeffeed87bc0f1` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:31.654122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822502e6b8d849bfb5eeffeed87bc0f1)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:31.657332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:32.339353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:32.340944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:32.342128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:32.376862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:32.378425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:32.386155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822502e6b8d849bfb5eeffeed87bc0f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:32.387076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822502e6b8d849bfb5eeffeed87bc0f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:32.423894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822502e6b8d849bfb5eeffeed87bc0f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:32.550206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1150/2314] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:19:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:32.573628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:32.575334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:32.718141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822502e6b8d849bfb5eeffeed87bc0f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:32.728913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-b7da105b-14d0-4d2a-9099-03ec8c7c0e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822502e6b8d849bfb5eeffeed87bc0f1'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:32.743301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1165/2315] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:19:31 2022] DELETE /identity/v3/users/822502e6b8d849bfb5eeffeed87bc0f1 => generated 0 bytes in 1120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:32.750021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:32.750598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:32.751127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:32.778771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79bf33-3aa7-42a2-8cb9-b4a5b63f38e3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:32.793949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:32.796715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:32.941341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1166/2316] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:32.956587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:32.956587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:32.956839 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:32.957248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:32.961673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:32.963884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:33.078823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09544d-8397-409b-8a5b-c6531c4ee0b9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:33.087305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:33.090862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:33.090862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:33.090862 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:33.096501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:33.100697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c793b4-335c-491c-bc97-908263c49a13 service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:33.113062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:33.117398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:33.132205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1151/2317] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:19:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 06:19:33.135554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1167/2318] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:19:33.847602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:33.932840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:33.933584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:33.934019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:34.017164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] MFA Rules not processed for user `2f141efe3bb74587b5ba13f0cd5d89db`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:34.019308 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:34.188579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1152/2319] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:34.194167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:34.195739 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:34.224816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a96832-2584-4ded-84dd-b51475b6f31b service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:34.238589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:34.242316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:34.245807 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:34.247607 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:34.248489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:34.250523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:34.488318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:34.491331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:34.510344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03daaeb-22e1-40cb-9889-09cce94487aa None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:34.532772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:34.534578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:34.537512 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:34.538005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] PATH_INFO: `/v3/projects/37ad375058384bd8b6d2e94711e733d0` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:34.541245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ad375058384bd8b6d2e94711e733d0)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:34.544151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:34.566324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:34.590520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1168/2320] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:34 2022] POST /identity/v3/projects => generated 377 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:34.590520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:34.734525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fbd87-8a98-423d-b868-f9bd10472b0c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:34.746342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:34.747174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:34.747519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:34.748125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:34.749153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:34.751254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:34.937384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ad375058384bd8b6d2e94711e733d0', 'request_id': 'req-2370c3fb-ce95-4743-b45f-e4b246ea88d5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:34.983643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1169/2321] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:34 2022] POST /identity/v3/users => generated 295 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:19:34.993747 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:35.014592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aa532-b61f-4598-80ba-6b2bd3bed8d4 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:35.079990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ad375058384bd8b6d2e94711e733d0', 'request_id': 'req-2370c3fb-ce95-4743-b45f-e4b246ea88d5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:35.103418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ad375058384bd8b6d2e94711e733d0', 'request_id': 'req-2370c3fb-ce95-4743-b45f-e4b246ea88d5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:35.127687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] The token cache is being invalidate because project 37ad375058384bd8b6d2e94711e733d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:19:35.128290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:35.134221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2370c3fb-ce95-4743-b45f-e4b246ea88d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:19:35.136444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1153/2322] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:19:34 2022] DELETE /identity/v3/projects/37ad375058384bd8b6d2e94711e733d0 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:35.141148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879ebd-c7c9-4194-acae-a691350e9c73 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.141450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879ebd-c7c9-4194-acae-a691350e9c73 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.141682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf879ebd-c7c9-4194-acae-a691350e9c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.149345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:35.150131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.150292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.150596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.151395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:35.155095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:35.181352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1170/2323] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:35.183353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:35.198113 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:35.218736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-cf879ebd-c7c9-4194-acae-a691350e9c73 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:35.241267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-cf879ebd-c7c9-4194-acae-a691350e9c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:35.251043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fffdbc-7e05-4589-b762-b4df4233cbf3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:35.258011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1154/2324] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:19:35.259236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:35.260314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.260601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.260872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] PATH_INFO: `/v3/projects/ec0cd181b35644639e7b0c20e865c6b6/users/0241c31fc7a0437cbe997b30cbc90fcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.277479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.382880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.382880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.384256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0cd181b35644639e7b0c20e865c6b6, user_id=0241c31fc7a0437cbe997b30cbc90fcf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:35.384256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:35.385019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:35.385019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:35.385019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1155/2325] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:35.386170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:35.406321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07892364-57c2-4209-94c1-f89dca7f60e6 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:35.437376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:35.438925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.439574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.440208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.441424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:19:35.442077 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:35.446028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:35.468096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1156/2326] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:19:35.476431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:35.493605 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447681d-732c-4cf4-a6aa-4636932c98cd admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:35.507079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:35.510066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:35.510542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:35.511655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:35.513004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:35.517443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:35.941292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1171/2327] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:35 2022] PUT /identity/v3/projects/ec0cd181b35644639e7b0c20e865c6b6/users/0241c31fc7a0437cbe997b30cbc90fcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:35.947102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:36.076425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5b4e2-9416-4295-b4c3-fb270f34766d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:36.083382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:36.084343 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:36.084732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:36.085129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:36.085905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:36.087679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:36.112302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1157/2328] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:19:35 2022] POST /identity/v3/projects => generated 363 bytes in 635 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:36.122070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:36.135132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1172/2329] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:36.137339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:36.151921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca815ecc-5312-4376-94e3-047ca88ab513 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:36.291366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:36.292815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:36.293273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:36.293740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] PATH_INFO: `/v3/projects/ec0cd181b35644639e7b0c20e865c6b6/users/0241c31fc7a0437cbe997b30cbc90fcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:36.296919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0cd181b35644639e7b0c20e865c6b6, user_id=0241c31fc7a0437cbe997b30cbc90fcf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:36.300061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:36.316833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1173/2330] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:36 2022] PUT /identity/v3/projects/ec0cd181b35644639e7b0c20e865c6b6/users/0241c31fc7a0437cbe997b30cbc90fcf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:19:36.357342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a927c1-9270-4f19-9975-e6ef66048a7a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:36.473317 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:36.474644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:36.475505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:36.475885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:36.476657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:36.487006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:36.738284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=17,keystone:SELECT=78,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:36.852180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1158/2331] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:19:36 2022] POST /identity/v3/users => generated 282 bytes in 737 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:19:36.856622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:36.874757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d2df1-02c3-48d1-8f74-389a23dccea1 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:36.887119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:36.887618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:36.889405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:36.889405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:36.891402 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:36.896853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:36.927660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:36.928706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1174/2332] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:36.941978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26e32e-f338-485d-bd0b-8671e10675ab admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:36.953840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:36.954771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:36.955478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:36.955478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:36.956132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:36.959148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:36.980610 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1159/2333] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:19:36.985417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:37.002679 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:37.007784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:37.020487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c20abd-c345-4b39-898e-cadb655534e3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:37.035802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:37.037322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:37.038038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:37.038501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] PATH_INFO: `/v3/projects/f2f19e3b987b4cf599476f4670c2e42d/users/9f033391f57849bd8c0c8f0190538134/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:37.082014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f19e3b987b4cf599476f4670c2e42d, user_id=9f033391f57849bd8c0c8f0190538134, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:37.098314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adf426-a595-446d-a207-a4223cedad37 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:37.221857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a211c-a1d0-473d-bc5a-6c3e78636691 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:37.234978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:37.236028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:37.236028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:37.236268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:37.247572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:37.276004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:37.277772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:37.291008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1175/2334] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:19:36 2022] PUT /identity/v3/projects/f2f19e3b987b4cf599476f4670c2e42d/users/9f033391f57849bd8c0c8f0190538134/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:19:37.300334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1160/2335] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:37.316795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bb32-809e-469f-bea8-f2f530c4a4da None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:37.317648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bb32-809e-469f-bea8-f2f530c4a4da None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:37.317648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0bb32-809e-469f-bea8-f2f530c4a4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:37.443145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c1d0bb32-809e-469f-bea8-f2f530c4a4da None None] MFA Rules not processed for user `9f033391f57849bd8c0c8f0190538134`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:37.447600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c1d0bb32-809e-469f-bea8-f2f530c4a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:37.621502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1176/2336] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:37.641671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:37.643406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:37.816680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fadea7-1696-4cc7-bca2-23c1b15dabfc service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:37.969418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:37.970548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:37.971014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:37.971453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:38.008023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:38.009765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:38.081298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1161/2337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:46.102845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:47.615018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:51.149690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a53c-af7a-4e4e-a5f1-bc7ec820215c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:51.149690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a53c-af7a-4e4e-a5f1-bc7ec820215c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:51.149690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006a53c-af7a-4e4e-a5f1-bc7ec820215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:51.149690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c006a53c-af7a-4e4e-a5f1-bc7ec820215c None None] MFA Rules not processed for user `0241c31fc7a0437cbe997b30cbc90fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:51.149690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c006a53c-af7a-4e4e-a5f1-bc7ec820215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:51.312239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1177/2338] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:19:51.331242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:19:51.333800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:51.623449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f75a16-00cb-4baf-929f-455e99ed623b service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:19:51.652593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:19:51.653485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:51.653832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:51.654120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:51.657193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:51.877495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:19:51.879231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d547d-a014-405a-89d9-116e17d39bb3 service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:19:51.898337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1162/2339] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:19:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:19:55.273103 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:56.457199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:19:57.092645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:19:57.093548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:19:57.094016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:19:57.126443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] MFA Rules not processed for user `61318fafe2a1469c884b5290fe4ba366`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:19:57.128598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:19:57.226669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1178/2340] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:19:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:01.418901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:02.357742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:03.368978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:03.813616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:05.180943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:07.632921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:10.226538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:10.933037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4c5-45bf-4a1d-a73a-2091cbaa333a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:10.933515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4c5-45bf-4a1d-a73a-2091cbaa333a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:10.934165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41c4c5-45bf-4a1d-a73a-2091cbaa333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:10.965751 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7d41c4c5-45bf-4a1d-a73a-2091cbaa333a None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:10.968016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7d41c4c5-45bf-4a1d-a73a-2091cbaa333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:11.114574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1163/2341] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:20:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:11.153604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:20:11.157637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:11.246678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:11.272353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658323b-cbee-4e5e-82a2-7533021f2e1e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:20:11.280767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:20:11.281519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:11.281874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:11.282319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:11.299555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:20:11.301636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68436489-728f-4148-ba4d-22f56c173d06 service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:20:11.312415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1179/2342] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:20:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:20:18.571493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:20.150148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:20.981288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549290f5-0ebb-495d-a353-d773fe168682 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:20.982070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549290f5-0ebb-495d-a353-d773fe168682 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:20.982309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549290f5-0ebb-495d-a353-d773fe168682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:21.044949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-549290f5-0ebb-495d-a353-d773fe168682 None None] MFA Rules not processed for user `2f141efe3bb74587b5ba13f0cd5d89db`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:21.046831 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-549290f5-0ebb-495d-a353-d773fe168682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:21.093291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1164/2343] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:20:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:24.677881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:25.661430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:27.282232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:29.243645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:29.245003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:29.245466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:29.279010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] MFA Rules not processed for user `2cffd4ac2817448e94d2b32ac7ffcede`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:29.280413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:29.394821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1180/2344] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:20:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:29.784873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fae26c-2705-4cff-a556-cdc5d5bd9aef None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:29.786109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fae26c-2705-4cff-a556-cdc5d5bd9aef None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:29.786518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fae26c-2705-4cff-a556-cdc5d5bd9aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:29.847784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-23fae26c-2705-4cff-a556-cdc5d5bd9aef None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:29.853184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-23fae26c-2705-4cff-a556-cdc5d5bd9aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:29.875563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1165/2345] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:20:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:29.891713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:20:29.929828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11605e54-3c1e-4d61-98cf-67085a6f57db None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:20:29.935649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:20:29.936378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:29.939556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:29.939920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:29.951424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:20:29.953121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:20:29.974081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1181/2346] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:20:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 06:20:32.301124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:35.977267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:36.493318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abc16e-7ee1-47d3-976d-77fee66aedd4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:36.493811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abc16e-7ee1-47d3-976d-77fee66aedd4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:36.494464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abc16e-7ee1-47d3-976d-77fee66aedd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:36.548525 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-91abc16e-7ee1-47d3-976d-77fee66aedd4 None None] MFA Rules not processed for user `543a9cddd92a44c78c26e5b8400e0c98`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:36.556465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-91abc16e-7ee1-47d3-976d-77fee66aedd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:36.677351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1166/2347] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:20:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:20:41.069944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:49.620737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:50.355804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:20:50.495029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03155df-0349-44a6-abda-029c18b54c4a service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:20:50.507879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:20:50.508882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:50.509280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:50.509651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] PATH_INFO: `/v3/users/9f033391f57849bd8c0c8f0190538134` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:50.516618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f033391f57849bd8c0c8f0190538134)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:20:50.527376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:20:51.163894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f033391f57849bd8c0c8f0190538134'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:51.164727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f033391f57849bd8c0c8f0190538134'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:51.179281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f033391f57849bd8c0c8f0190538134'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:51.388711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f033391f57849bd8c0c8f0190538134'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:51.406304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f033391f57849bd8c0c8f0190538134'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:51.426184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1182/2348] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:20:50 2022] DELETE /identity/v3/users/9f033391f57849bd8c0c8f0190538134 => generated 0 bytes in 1065 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:20:51.428102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:51.428835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:51.429045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:51.461999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:20:51.463633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:51.563210 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1167/2349] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:20:51.580020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:20:51.581947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:51.709557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3704e3-0721-4d74-a2b2-68aeebd77f4e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:20:51.713714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:20:51.714352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:51.714838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:51.715270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:51.729307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:51.735248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:20:51.737737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:20:51.755664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1183/2350] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:20:52.741479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:20:52.741479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:20:52.784431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8020330-e7cd-42cf-98fa-740e8de16cb4 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:20:52.799648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:20:52.801560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:20:52.802124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:20:52.806706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] PATH_INFO: `/v3/projects/f2f19e3b987b4cf599476f4670c2e42d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:20:52.810588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f19e3b987b4cf599476f4670c2e42d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:20:52.821897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:20:53.017910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f19e3b987b4cf599476f4670c2e42d', 'request_id': 'req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:53.431289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f19e3b987b4cf599476f4670c2e42d', 'request_id': 'req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:53.438001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f19e3b987b4cf599476f4670c2e42d', 'request_id': 'req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:53.452418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] The token cache is being invalidate because project f2f19e3b987b4cf599476f4670c2e42d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:20:53.453117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:53.453773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bf5eacfc-ac2a-4085-aab3-2c24e7342ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:20:53.455644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1168/2351] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:20:52 2022] DELETE /identity/v3/projects/f2f19e3b987b4cf599476f4670c2e42d => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:20:58.540728 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:20:59.905929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:00.436038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:00.448137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215925c-168a-4541-b981-2bbecb6a9edb service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:00.510325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:00.510974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:00.511263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:00.511542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:00.513302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:00.587025 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:00.588395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac403e-eb3a-40ab-a68a-eec7c26b19f1 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:00.599736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1184/2352] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:21:01.486912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:01.487681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:01.487681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:01.489843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1169/2353] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:21:01.496965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0279-0061-4b82-88cf-d1380b879ecd None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:01.499447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0279-0061-4b82-88cf-d1380b879ecd None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:01.500161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee0279-0061-4b82-88cf-d1380b879ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:01.530689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-01ee0279-0061-4b82-88cf-d1380b879ecd None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:01.532168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-01ee0279-0061-4b82-88cf-d1380b879ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:01.622066 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1185/2354] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:21:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:21:01.636321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:01.636321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:01.699848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603701-d7ee-4583-9d32-ac628f3bd7b2 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:01.704901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:01.704901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:01.705599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:01.705599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:01.725043 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:01.726826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2901486-e0ea-42ee-afcd-da092c4c28f6 service glance] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:01.742689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1170/2355] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:02.150053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11468259-f88b-481d-91bb-16786898b206 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.150053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11468259-f88b-481d-91bb-16786898b206 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.152311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11468259-f88b-481d-91bb-16786898b206 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.152311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1186/2356] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:21:02.157431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dfa4-134e-47d7-8d38-a5180a5bbc5c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.157891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dfa4-134e-47d7-8d38-a5180a5bbc5c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.158287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dfa4-134e-47d7-8d38-a5180a5bbc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.181941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9da0dfa4-134e-47d7-8d38-a5180a5bbc5c None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:02.186508 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9da0dfa4-134e-47d7-8d38-a5180a5bbc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:02.193205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1171/2357] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:21:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:02.212938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11468259-f88b-481d-91bb-16786898b206 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:02.215100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-11468259-f88b-481d-91bb-16786898b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:02.239883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11468259-f88b-481d-91bb-16786898b206 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:02.245340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:02.246677 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.247090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.247669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.259020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:02.260516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f67113-50a0-4396-9e47-0a490672e619 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:02.271151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1187/2358] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:02.382992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.383420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.383942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.386051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1172/2359] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:21:02.397425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.398144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.398657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.422913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:02.424597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:02.440178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1188/2360] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:21:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:02.477593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:02.480138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:02.498048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140ce0f-9c57-4b84-b232-5ad40448adcb None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:02.504887 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:02.506674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:02.506674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:02.507213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:02.527044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:02.530480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8daa-6375-442e-96c1-9c9dc5ab1c8f service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:02.557337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1173/2361] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:03.208468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:03.219692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa53f1-309f-4c6a-a818-f526f863f870 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:03.225111 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:03.225778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:03.226060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:03.226334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:03.228138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:03.321759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:03.323325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:03.338362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1189/2362] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:05.928664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:07.535212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:11.937760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01778c8b-2abc-432d-95e7-2177ba4a4570 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:11.938282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01778c8b-2abc-432d-95e7-2177ba4a4570 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:11.938282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01778c8b-2abc-432d-95e7-2177ba4a4570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:11.961593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-01778c8b-2abc-432d-95e7-2177ba4a4570 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:11.962880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-01778c8b-2abc-432d-95e7-2177ba4a4570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:12.022251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1174/2363] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:12.045141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:12.046609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:12.122280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d9c24-3134-4903-8be1-411360f94e43 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:12.127587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:12.128744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:12.129030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:12.129317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:12.137854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:12.139516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:12.148622 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1190/2364] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:21:15.209379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:16.054020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:18.191339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:18.597891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:20.013878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:20.015092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:20.015092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:20.041143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:20.043299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:20.055456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1175/2365] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:21:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:20.067690 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:20.083672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea1512-316b-4bfe-ab7c-7e85bcc8e426 service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:20.092170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:20.092962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:20.097735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:20.097735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:20.105340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:20.107017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:20.117045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1191/2366] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:21:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:21:22.579563 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:25.214783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:25.718604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:25.739794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7432c8-ae54-495e-8f6f-593401b6fb65 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:25.765225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:25.770204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:25.770709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:25.771111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:25.775364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:25.806480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:25.808917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8523662-adc0-4a44-9a01-3f1f00c9a9c9 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:25.835848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1176/2367] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:25.994951 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:26.012947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3b88e-97ec-47a8-9d1a-80548c47ccfe service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:26.025319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:26.026689 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:26.027640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:26.028455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:26.033116 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:26.054804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:26.057761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:26.090381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1192/2368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:26.724313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:27.202699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:31.053551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:31.871373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:33.323956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:33.712759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:35.473159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:36.420891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:37.314231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:37.314866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:37.315351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:37.355220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] MFA Rules not processed for user `8d154e96376a49918658b0c340c722e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:37.357248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:37.468793 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1177/2369] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:21:37.482250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:37.785629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:38.086115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:38.088390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:38.105639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf0f0-62e8-48a1-931b-91b56749a030 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:38.113177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:38.114131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:38.114461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:38.114780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:38.116589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:38.131785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:38.133413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c206a-82cd-437d-a1a2-5e221f7b1b38 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:38.150711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1193/2370] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:21:38.641856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:39.972479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:39.974226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:40.046065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a253a-a56c-4a2a-a045-052d28526ba0 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:40.053758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:40.054377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:40.054686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:40.054962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:40.056982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:40.136854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:40.138412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:40.171012 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1178/2371] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:40.526229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:41.009558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:41.708255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:41.708651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:41.709105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:41.742597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:41.744030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:41.753585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1194/2372] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:21:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:41.771296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:41.789592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4f208-6811-4847-8ad3-9d3e5c24c985 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:41.796130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:41.801794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:41.801794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:41.801794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:41.808181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:41.810139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:41.821503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1179/2373] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:21:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:21:43.614955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:45.032123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:46.986720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:46.989178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:47.063213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f4786-db97-4c00-b76f-21ba3e7bf1a7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:47.068021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:47.068659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:47.070332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:47.070332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] PATH_INFO: `/v3/users/2cffd4ac2817448e94d2b32ac7ffcede` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:47.070532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cffd4ac2817448e94d2b32ac7ffcede)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:47.072459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:47.442851 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cffd4ac2817448e94d2b32ac7ffcede'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:47.444157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cffd4ac2817448e94d2b32ac7ffcede'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:47.466871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cffd4ac2817448e94d2b32ac7ffcede'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:47.500032 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:47.614993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cffd4ac2817448e94d2b32ac7ffcede'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:47.631642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8ab2490a-81fb-41c9-b131-25fb20d879b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cffd4ac2817448e94d2b32ac7ffcede'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:47.645591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1195/2374] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:21:46 2022] DELETE /identity/v3/users/2cffd4ac2817448e94d2b32ac7ffcede => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:21:48.169687 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:48.188137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdd6e3-8395-4e3c-8fe9-e97d1eb53c47 service swift] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:48.287835 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:48.290240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:48.290969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:48.291500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] PATH_INFO: `/v3/projects/b38f9319c7534904ac58d1d9f9048b91` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:48.294366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38f9319c7534904ac58d1d9f9048b91)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:48.299347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:48.547092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38f9319c7534904ac58d1d9f9048b91', 'request_id': 'req-5597b432-271d-4311-9520-20fe7e8f2462'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:48.716217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38f9319c7534904ac58d1d9f9048b91', 'request_id': 'req-5597b432-271d-4311-9520-20fe7e8f2462'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:48.724013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38f9319c7534904ac58d1d9f9048b91', 'request_id': 'req-5597b432-271d-4311-9520-20fe7e8f2462'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:48.739503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] The token cache is being invalidate because project b38f9319c7534904ac58d1d9f9048b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:21:48.740974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:48.740974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5597b432-271d-4311-9520-20fe7e8f2462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:48.745323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1180/2375] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:21:48 2022] DELETE /identity/v3/projects/b38f9319c7534904ac58d1d9f9048b91 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:21:48.750564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:48.751155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:48.751577 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:48.776454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:48.777834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:48.871516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1196/2376] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:48.880065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:48.880352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:48.880623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:48.902804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:48.904084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:48.918574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1181/2377] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:48.923730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:48.948010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044cda9b-64de-446b-b6c4-25f631369604 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:48.961315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:48.962132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:48.970095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:48.970095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:48.970095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:21:48.970095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:48.970095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:48.980687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1197/2378] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:21:48.986855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.005264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1192b8-95f9-402f-8adc-da687620b569 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:49.011121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:49.011573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.011914 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.017667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.017667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:49.017667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:49.197169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1182/2379] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:21:48 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:21:49.198224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.217629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c74bc-da3a-4684-ba6f-d9c47b9f8f20 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:49.329232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:49.330234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.331290 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.332878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.334459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:49.337311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717140e5-f038-427c-8f0c-efdc6ef67f7d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:49.645067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1198/2380] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:21:49 2022] POST /identity/v3/users => generated 292 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:21:49.650345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.668495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce95f6-673c-40b4-8aba-3f7769f769fb admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:49.695368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:49.696593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.697079 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.700910 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.702241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:49.706007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:49.727509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.727900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.728277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.757160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:49.759312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:49.768141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1183/2381] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:21:49.769320 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.807329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e6a0-0c33-4af5-8b74-cf24bd7e17ea admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:49.832476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:49.833104 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.833489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.833773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.834432 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:49.836146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:49.855537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1199/2382] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:21:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:21:49.868130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.871560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1184/2383] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:21:49.874729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:49.877311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:49.896822 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b35db8-f2ff-46b5-b755-9d0753385057 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:49.905034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:49.913504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:49.913504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:49.913504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] PATH_INFO: `/v3/projects/684358fb8c42489b89de8e8e863cfe00/users/64c6b50596374543bdd8aa51deb88e35/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:49.945167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=684358fb8c42489b89de8e8e863cfe00, user_id=64c6b50596374543bdd8aa51deb88e35, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:49.948517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:50.067449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bd50-99cc-4835-8d91-bcfab70445de admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:50.077586 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:50.079013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:50.080031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:50.080612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:50.083752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:50.100744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:50.104619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdad3b1-f212-46f2-9559-98270c812f47 service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:50.125396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1185/2384] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:21:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:21:50.133078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1200/2385] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:21:49 2022] PUT /identity/v3/projects/684358fb8c42489b89de8e8e863cfe00/users/64c6b50596374543bdd8aa51deb88e35/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:21:50.146380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:50.147749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:50.148119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:50.196344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] MFA Rules not processed for user `64c6b50596374543bdd8aa51deb88e35`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:50.197855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:50.311003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1186/2386] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:50.325903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:50.329202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:50.501435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aa5f6-9f36-4f19-8f64-fc2a1f464d43 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:50.509684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:50.510570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:50.510904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:50.511180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:50.532949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:50.536803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:50.570926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1201/2387] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:51.091243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:52.856403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:52.864247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:52.978664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12ee49-eb87-46f5-8e74-0797d8781676 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:52.983691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:52.984438 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:52.984719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:52.984993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:52.992360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:53.010975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:53.014753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19aebe0-769f-438e-9123-4f5a559df2fd service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:53.034088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1187/2388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:21:56.419716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:56.543657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76600a-37a5-4401-9270-63a14fe3275d service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:56.549757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:56.550466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:56.550861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:56.551133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] PATH_INFO: `/v3/users/64c6b50596374543bdd8aa51deb88e35` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:56.552944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c6b50596374543bdd8aa51deb88e35)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:56.556442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:57.279301 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c6b50596374543bdd8aa51deb88e35'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:57.280370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c6b50596374543bdd8aa51deb88e35'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:57.305368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c6b50596374543bdd8aa51deb88e35'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:57.487321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c6b50596374543bdd8aa51deb88e35'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:57.496785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c6b50596374543bdd8aa51deb88e35'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:57.507940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1202/2389] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:21:56 2022] DELETE /identity/v3/users/64c6b50596374543bdd8aa51deb88e35 => generated 0 bytes in 1089 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:21:57.512683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:57.513487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:57.513487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:57.537858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:57.539178 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:57.607754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1188/2390] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:57.618713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:57.620422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:57.630794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:57.686795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd2db8-1e5b-41ad-9001-593286c05af1 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:57.691187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:57.691895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:57.692200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:57.692471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:57.703338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:57.706403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416af52-d401-41df-9e5c-e5b5e7bcac01 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:57.718090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1203/2391] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:21:58.292598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:21:58.296920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:58.299036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:58.324797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ff274-fc57-4578-b597-3a8955ba99d9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:58.335397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:58.336149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:58.336513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:58.336693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] PATH_INFO: `/v3/projects/684358fb8c42489b89de8e8e863cfe00` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:58.338273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=684358fb8c42489b89de8e8e863cfe00)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:58.340415 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:58.666180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684358fb8c42489b89de8e8e863cfe00', 'request_id': 'req-3a7a7807-9841-458e-99ad-505cbe1cc046'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:58.921689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684358fb8c42489b89de8e8e863cfe00', 'request_id': 'req-3a7a7807-9841-458e-99ad-505cbe1cc046'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:58.932800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684358fb8c42489b89de8e8e863cfe00', 'request_id': 'req-3a7a7807-9841-458e-99ad-505cbe1cc046'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:58.946477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] The token cache is being invalidate because project 684358fb8c42489b89de8e8e863cfe00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:21:58.946962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:58.947674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a7a7807-9841-458e-99ad-505cbe1cc046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:21:58.949236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1189/2392] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:21:58 2022] DELETE /identity/v3/projects/684358fb8c42489b89de8e8e863cfe00 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:21:58.954472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:58.955176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:58.955176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:58.980039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:58.981311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:59.048381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1204/2393] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:21:59.058496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.058888 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.059186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.085908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:21:59.087222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:21:59.108913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1190/2394] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:21:59.113272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.125466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558affa-3c3d-478b-a391-37abdc0274e0 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.130416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.131099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.131387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.131668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.132373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:21:59.132687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.134336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.150073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1205/2395] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:21:59.156130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.168576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce0b44-9443-4bdc-90ba-837492088370 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.174220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.175100 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.175335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.175638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.176405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.178292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.342805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1191/2396] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:21:59 2022] POST /identity/v3/projects => generated 389 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:21:59.347019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.359163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1464d7e-1c0e-46c6-98fd-288a3fe846e8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.425634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.426181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.426464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.426777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.427423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.429530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.642395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1206/2397] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:21:59 2022] POST /identity/v3/users => generated 295 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:21:59.646841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.657693 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f6ff3-7e0a-4384-ae5e-d82f4f11a49c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.662857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.663498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.663754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.664019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.664666 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.666271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.678688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1192/2398] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:21:59.682805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.699895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e45ad-e295-4766-9a64-531a80c9aa7b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.700223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.700838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.701120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.701392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.702039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.703692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f87e35-a5ce-4a56-8d26-c1d869e45a00 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.719089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1207/2399] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:21:59.720422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:21:59.733184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d163f-0112-45d8-9de1-414294a884d7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:21:59.742055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:21:59.743206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.743702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.752668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] PATH_INFO: `/v3/projects/bd9e004956a441c49c66a18e558c4c20/users/fe772eff1a9844d6b2628578b0ef6b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:21:59.762698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd9e004956a441c49c66a18e558c4c20, user_id=fe772eff1a9844d6b2628578b0ef6b2f, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:21:59.765455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:21:59.985771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1193/2400] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:21:59 2022] PUT /identity/v3/projects/bd9e004956a441c49c66a18e558c4c20/users/fe772eff1a9844d6b2628578b0ef6b2f/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:21:59.998638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2132a7-7b2e-4d0a-9834-6c94072c89a4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:21:59.999186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2132a7-7b2e-4d0a-9834-6c94072c89a4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:21:59.999295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2132a7-7b2e-4d0a-9834-6c94072c89a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:00.036089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3f2132a7-7b2e-4d0a-9834-6c94072c89a4 None None] MFA Rules not processed for user `fe772eff1a9844d6b2628578b0ef6b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:00.042636 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3f2132a7-7b2e-4d0a-9834-6c94072c89a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:00.136101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1208/2401] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:21:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:00.144445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:00.146527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:00.226488 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670588-e575-4acd-9ab2-4dc59d0c37bf admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:00.232108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:00.233136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:00.233136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:00.233325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:00.250356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:00.251867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:00.266202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1194/2402] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:22:01.462586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:01.463097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:01.463424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:01.486074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] MFA Rules not processed for user `1d22ea3fb8fa4bd99537fb83279b2d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:01.487947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:01.549263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1209/2403] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:02.129021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:02.130476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:02.190364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb3165-d031-4982-8ceb-98b9ec54c2c4 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:02.194464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:02.195110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:02.195414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:02.195689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:02.197449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:02.208426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:02.209869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:02.218646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1195/2404] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:02.721223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:02.722788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:02.807298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612e0de-875a-40d0-8750-f8d8498f5923 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:02.812716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:02.817787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:02.817787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:02.817787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:02.817787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:02.829131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:02.830821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:02.841666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1210/2405] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:03.201870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:03.960777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:07.727565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:09.292912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:09.967140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:11.076027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:11.424595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:11.426350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:11.509129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acc72a-f89d-4a1a-9f47-22d3bd2469a3 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:11.519935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:11.520570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:11.520851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:11.521124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:11.523125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:11.536863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:11.538560 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:11.552037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1196/2406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:12.669561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:13.321697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:16.527927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:17.978535 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:19.901389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:20.124641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:20.138949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0de6c-4db0-4d46-87bd-64a7e2159137 service placement] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:20.146147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:20.146910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:20.147296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:20.152042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:20.152042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:20.169983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:20.174451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:20.188334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1211/2407] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:20.401354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:22.128776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:23.183853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:23.327674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:24.144660 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:25.401700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:26.224917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:26.646788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:27.829952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:27.839568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa14c2f-0f88-405b-afba-41f12070193b service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:27.847511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:27.848379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:27.848688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:27.849016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:27.851217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:27.867926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:27.868939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7f098-59ca-45be-8bda-102d85d771bb service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:27.883859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1197/2408] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:28.321854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:29.660841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:29.732458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4829cc-c1fe-4772-a929-802b7807a47e service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:29.739521 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:29.743881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:29.743881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:29.743881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] PATH_INFO: `/v3/users/fe772eff1a9844d6b2628578b0ef6b2f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:29.743881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe772eff1a9844d6b2628578b0ef6b2f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:29.744366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:30.139296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe772eff1a9844d6b2628578b0ef6b2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:30.139933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe772eff1a9844d6b2628578b0ef6b2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:30.153276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe772eff1a9844d6b2628578b0ef6b2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:30.253243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe772eff1a9844d6b2628578b0ef6b2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:30.255933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:30.263232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe772eff1a9844d6b2628578b0ef6b2f'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:30.270281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1212/2409] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:22:29 2022] DELETE /identity/v3/users/fe772eff1a9844d6b2628578b0ef6b2f => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:22:30.276430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:30.276952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:30.276952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:30.300511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:30.301811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:30.374127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1198/2410] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:30.384221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:30.385684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:30.445644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cb25e-bb32-47ca-af84-85ae85bb886a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:30.449263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:30.449872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:30.450157 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:30.450435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:30.460516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:30.461961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:30.473215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1213/2411] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:31.056010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:31.057669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:31.076023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a9078-b5b9-4f10-a7df-f0cbf4238e49 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:31.082443 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:31.083162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:31.083434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:31.083806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] PATH_INFO: `/v3/projects/bd9e004956a441c49c66a18e558c4c20` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:31.084938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd9e004956a441c49c66a18e558c4c20)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:31.087115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:31.260337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd9e004956a441c49c66a18e558c4c20', 'request_id': 'req-f60abffc-68f6-4cab-bc52-576d9c467bc4'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:31.366571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd9e004956a441c49c66a18e558c4c20', 'request_id': 'req-f60abffc-68f6-4cab-bc52-576d9c467bc4'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:31.374151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd9e004956a441c49c66a18e558c4c20', 'request_id': 'req-f60abffc-68f6-4cab-bc52-576d9c467bc4'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:31.390275 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] The token cache is being invalidate because project bd9e004956a441c49c66a18e558c4c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:22:31.390805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:31.391557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:31.394084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1199/2412] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:22:31 2022] DELETE /identity/v3/projects/bd9e004956a441c49c66a18e558c4c20 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:22:32.578592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:34.092523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:36.054199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:37.441553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:38.876252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:39.048161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:39.048161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:39.062016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ffcc9-4645-4217-a1c8-c6de520cad30 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:39.127669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:39.128448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:39.128812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:39.129072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] PATH_INFO: `/v3/users/543a9cddd92a44c78c26e5b8400e0c98` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:39.130253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=543a9cddd92a44c78c26e5b8400e0c98)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:39.132483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:39.472616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '543a9cddd92a44c78c26e5b8400e0c98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:39.473360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '543a9cddd92a44c78c26e5b8400e0c98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:39.488614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '543a9cddd92a44c78c26e5b8400e0c98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:39.629742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '543a9cddd92a44c78c26e5b8400e0c98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:39.637464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-a1f28a8a-3663-4063-91e5-2478813b9294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '543a9cddd92a44c78c26e5b8400e0c98'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:39.644133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1214/2413] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:22:39 2022] DELETE /identity/v3/users/543a9cddd92a44c78c26e5b8400e0c98 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:22:39.980319 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:39.981939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:39.995195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:39.996841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60abffc-68f6-4cab-bc52-576d9c467bc4 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:40.071286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:40.071933 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.072213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.072491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] PATH_INFO: `/v3/projects/7e90dec3a44a4763b0045af64212fc15` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.073656 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e90dec3a44a4763b0045af64212fc15)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:40.075629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:40.231030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e90dec3a44a4763b0045af64212fc15', 'request_id': 'req-d9c92e86-c5f7-497c-912a-de8a321a1f2c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:40.429209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e90dec3a44a4763b0045af64212fc15', 'request_id': 'req-d9c92e86-c5f7-497c-912a-de8a321a1f2c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:40.440455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e90dec3a44a4763b0045af64212fc15', 'request_id': 'req-d9c92e86-c5f7-497c-912a-de8a321a1f2c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:40.447283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] The token cache is being invalidate because project 7e90dec3a44a4763b0045af64212fc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:22:40.447572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:40.448345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d9c92e86-c5f7-497c-912a-de8a321a1f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:40.449838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1200/2414] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:22:39 2022] DELETE /identity/v3/projects/7e90dec3a44a4763b0045af64212fc15 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:22:40.455377 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.456102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.456277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.477742 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:40.479009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:40.544504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1215/2415] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:40.552873 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.553160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.553469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.576199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:40.577434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:40.586235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1201/2416] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:40.590937 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:40.601738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd4477-f9cd-4121-a25c-e04c67461ec6 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:40.606288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:40.606943 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.607292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.607626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.609921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:22:40.609921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:40.610282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:40.623726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1216/2417] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:22:40.629627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:40.639464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee46b3e-ffa0-4e67-bef5-4a42ae5bb23b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:40.644954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:40.645582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.645862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.646135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.646867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:40.648790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:40.860469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1202/2418] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:22:40 2022] POST /identity/v3/projects => generated 397 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:22:40.865337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:40.875920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd39786-6403-44a8-b6f7-107fc96ecb45 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:40.935624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:40.937361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:40.937361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:40.937361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:40.937712 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:40.940061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:41.128122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1217/2419] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:22:40 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:22:41.135285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:41.142231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2b8fd-f0b6-4223-942d-9e6a03f760d7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:41.146944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:41.147570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:41.147857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:41.148130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:41.148775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:41.150387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:41.160810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1203/2420] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:22:41.164736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:41.176891 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e85-af23-4804-9009-b7c2617104a7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:41.181740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:41.182454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:41.182791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:41.192656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:41.192656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:41.192656 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca87455-f74f-4672-ad65-dcd37ae95f4d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:41.198721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1218/2421] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:22:41.203047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:41.214386 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df176660-5722-403e-96d8-914a6e067c4d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:41.218969 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:41.219616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:41.219894 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:41.220161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] PATH_INFO: `/v3/projects/af6b870c7cb249e281d01ee3ceaff5d0/users/448d458b35734478ae078fddba9470df/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:41.235485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6b870c7cb249e281d01ee3ceaff5d0, user_id=448d458b35734478ae078fddba9470df, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:41.239619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:41.355228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1204/2422] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:22:41 2022] PUT /identity/v3/projects/af6b870c7cb249e281d01ee3ceaff5d0/users/448d458b35734478ae078fddba9470df/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:22:41.367718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:41.368010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:41.368412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:41.400001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] MFA Rules not processed for user `448d458b35734478ae078fddba9470df`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:41.401667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:41.470264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1219/2423] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:22:41.482524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:41.484156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:41.546791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c846142-70fb-4eae-b228-d4663279c954 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:41.551190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:41.551816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:41.552097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:41.552371 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:41.566126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:41.566433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27689338-82b9-494c-b2b0-2ada5b94cd1f service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:41.577788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1205/2424] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:42.008467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:42.337902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:46.532249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:46.698028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:47.943804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:48.069275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:49.841238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:50.390092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:51.820592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:51.903542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3ef45-7771-4af0-9b04-fddd5d6bc188 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:51.910465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:51.911499 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:51.912081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:51.912352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] PATH_INFO: `/v3/users/448d458b35734478ae078fddba9470df` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:51.914012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=448d458b35734478ae078fddba9470df)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:51.917602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:52.156135 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448d458b35734478ae078fddba9470df'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:52.157033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448d458b35734478ae078fddba9470df'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:52.171944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448d458b35734478ae078fddba9470df'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:52.303346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448d458b35734478ae078fddba9470df'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:52.315030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448d458b35734478ae078fddba9470df'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:52.317874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1220/2425] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:22:51 2022] DELETE /identity/v3/users/448d458b35734478ae078fddba9470df => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:22:52.322423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:52.322944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:52.323224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:52.345031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:52.346393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:52.395342 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:52.427955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1206/2426] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:52.437999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:52.440510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:52.503392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe2bdf-b778-462f-868c-2d763d2490fa admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:52.509030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:52.509877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:52.510221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:52.510643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:52.521533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:52.523039 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46916399-591e-4290-b1eb-d051f0fb2695 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:52.537392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1221/2427] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:52.886568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:53.170110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:53.172080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:53.197316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8b100-9432-4a71-ba52-a0c7315a5f09 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:53.204861 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:53.207070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:53.207680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:53.208241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] PATH_INFO: `/v3/projects/af6b870c7cb249e281d01ee3ceaff5d0` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:53.210343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6b870c7cb249e281d01ee3ceaff5d0)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:53.213035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:53.406115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6b870c7cb249e281d01ee3ceaff5d0', 'request_id': 'req-773e5d47-3c5e-4733-8f1d-64303457fb05'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:53.567869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6b870c7cb249e281d01ee3ceaff5d0', 'request_id': 'req-773e5d47-3c5e-4733-8f1d-64303457fb05'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:53.576239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6b870c7cb249e281d01ee3ceaff5d0', 'request_id': 'req-773e5d47-3c5e-4733-8f1d-64303457fb05'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:53.591654 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] The token cache is being invalidate because project af6b870c7cb249e281d01ee3ceaff5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:22:53.592010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:53.592694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-773e5d47-3c5e-4733-8f1d-64303457fb05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:22:53.594340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1207/2428] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:22:53 2022] DELETE /identity/v3/projects/af6b870c7cb249e281d01ee3ceaff5d0 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:22:53.599986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:53.602084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:53.602084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:53.629588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:53.631730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:53.744796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1222/2429] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:53.752579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:53.753124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:53.753651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:53.776213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:53.777725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:53.787389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1208/2430] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:53.791224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:53.803528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46122c-d2d3-4c97-ba9f-53a5456ff2c7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:53.808956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:53.809792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:53.810229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:53.810685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:53.812196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:22:53.812657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:53.820352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:53.839372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1223/2431] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:22:53.843587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:53.857347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be62a0c-51d0-43c8-b3db-ebafeeb19696 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:53.871220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:53.871747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:53.872211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:53.872948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:53.874040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:53.877049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.010517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:54.010796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1209/2432] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:22:53 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:22:54.030141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab111451-f219-44a5-bd7a-ae05600e2a88 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:54.123264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:54.124049 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.124339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.124602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.125270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:54.128378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.265485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1224/2433] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:22:54 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:22:54.272179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:54.284707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0eec04-2e12-4dc1-aaa8-56c535309bbf admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:54.289932 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:54.290935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.291456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.291865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.292694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:54.294480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.312913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1210/2434] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:22:54.316369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:54.326884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32af4-7dd0-4458-9b03-34c3d8546541 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:54.332590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:54.333249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.333634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.333933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.334586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:54.336372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983f534-ae83-43b8-9799-404a588d8000 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.352502 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1225/2435] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:22:54.358427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:54.369694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62fa7-9872-4aad-bb7d-ed7fdb0d5d44 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:54.396472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:54.397109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.397389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.397665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] PATH_INFO: `/v3/projects/52e8949a7d05434396f8e8631f2cdb72/users/a9ebeec374454468aae1d2b5fde21648/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.415966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e8949a7d05434396f8e8631f2cdb72, user_id=a9ebeec374454468aae1d2b5fde21648, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:54.418808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.424795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:54.780493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1211/2436] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:22:54 2022] PUT /identity/v3/projects/52e8949a7d05434396f8e8631f2cdb72/users/a9ebeec374454468aae1d2b5fde21648/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:22:54.786679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cb39-3a30-4b01-97ca-8938792953d1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.787171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cb39-3a30-4b01-97ca-8938792953d1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.787171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cb39-3a30-4b01-97ca-8938792953d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.809120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-3ad8cb39-3a30-4b01-97ca-8938792953d1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:22:54.810438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3ad8cb39-3a30-4b01-97ca-8938792953d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:54.878193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1226/2437] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:22:54.893417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:22:54.895218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:22:54.920928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:54.959000 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6eaee-6b10-4509-8997-1c876e4b5129 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:22:54.962828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:22:54.963459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:22:54.963739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:22:54.964013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:22:54.976004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:22:54.977467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:22:54.989852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1212/2438] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:22:57.281836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:58.069584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:22:59.955064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:01.040895 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:02.716360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3ba10-ebdf-4884-886a-467ab43f78b0 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:02.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3ba10-ebdf-4884-886a-467ab43f78b0 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:02.716908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3ba10-ebdf-4884-886a-467ab43f78b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:02.738167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a0d3ba10-ebdf-4884-886a-467ab43f78b0 None None] MFA Rules not processed for user `660c6df77d0a4750aac30345d6768526`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:02.743445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a0d3ba10-ebdf-4884-886a-467ab43f78b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:02.841566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1227/2439] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:03.011317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:03.456236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:03.477096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb77e8e-c113-43b9-8363-7e4c58a032a9 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:03.498247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:03.498926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:03.499716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:03.500025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:03.503592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:03.516498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:03.532882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:03.557163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1213/2440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:23:03.789609 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:04.569277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539837-3268-423b-b517-35ee6934a368 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:04.569823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539837-3268-423b-b517-35ee6934a368 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:04.570044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539837-3268-423b-b517-35ee6934a368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:04.607924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9a539837-3268-423b-b517-35ee6934a368 None None] MFA Rules not processed for user `a9ebeec374454468aae1d2b5fde21648`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:04.609564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9a539837-3268-423b-b517-35ee6934a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:04.710176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1228/2441] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:23:04.715620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:04.729389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c46ecf-1d29-4329-aa46-0af072981366 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:04.734494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:04.735303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:04.735773 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:04.736231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:04.737052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:04.739482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:05.126989 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1214/2442] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:04 2022] POST /identity/v3/projects => generated 373 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:23:05.128316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539837-3268-423b-b517-35ee6934a368 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:05.141841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539837-3268-423b-b517-35ee6934a368 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:05.227458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:05.228213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:05.228510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:05.228795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:05.229465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:05.231497 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:05.498849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1229/2443] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:05 2022] POST /identity/v3/users => generated 293 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:05.503288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:05.516480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f4da7-c03f-4cda-a5b4-a8fbf7b7a4a0 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:05.522956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:05.522956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:05.523271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:05.523496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:05.524111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:05.525734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:05.541603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1215/2444] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:05.543282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:05.556418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e050b47-5570-41d9-80a6-c56c6b95b434 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:05.562421 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:05.563330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:05.563959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:05.564364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] PATH_INFO: `/v3/projects/03e76ed17eff49e289513ca492f561b9/users/415da9e74d4a495b907572f9131e84c0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:05.584893 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e76ed17eff49e289513ca492f561b9, user_id=415da9e74d4a495b907572f9131e84c0, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:05.587949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:05.858474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1230/2445] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:05 2022] PUT /identity/v3/projects/03e76ed17eff49e289513ca492f561b9/users/415da9e74d4a495b907572f9131e84c0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:05.863213 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:05.874466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8482b-681a-4a78-978d-fd148fd726b2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:05.945433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:05.945958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:05.946240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:05.946517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:05.947214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:05.948870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:05.963629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1216/2446] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:05.964950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:05.975965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d02d-c6d2-40fa-8a2a-84a8202257f7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:05.981598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:05.981901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:05.981999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:05.982231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] PATH_INFO: `/v3/projects/03e76ed17eff49e289513ca492f561b9/users/415da9e74d4a495b907572f9131e84c0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:05.984424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e76ed17eff49e289513ca492f561b9, user_id=415da9e74d4a495b907572f9131e84c0, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:05.987730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173652ce-ed95-4de2-bdd0-8e6677f0acc0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:06.002958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1231/2447] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:05 2022] PUT /identity/v3/projects/03e76ed17eff49e289513ca492f561b9/users/415da9e74d4a495b907572f9131e84c0/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:06.042010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:07.029607 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:08.657896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:09.921137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:09.993945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c01b2c-91f7-4823-b3bc-776968913ccd admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:10.081249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:10.081934 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:10.082228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:10.082520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:10.084653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:10.187729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:10.195393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:10.218547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1217/2448] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:10.259299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:10.519435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:12.006662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:15.440274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:15.709349 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090dbe-d155-4a3c-bf9c-9ec1c0ccefa1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:15.709938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090dbe-d155-4a3c-bf9c-9ec1c0ccefa1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:15.709938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090dbe-d155-4a3c-bf9c-9ec1c0ccefa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:15.739598 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-35090dbe-d155-4a3c-bf9c-9ec1c0ccefa1 None None] MFA Rules not processed for user `415da9e74d4a495b907572f9131e84c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:15.740867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-35090dbe-d155-4a3c-bf9c-9ec1c0ccefa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:15.820552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1232/2449] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:15.977710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:15.979074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:16.002572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:16.004319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:16.004319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:16.040220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:16.041875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:16.080992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736feec2-da6e-4c95-81ea-704cb06c3281 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:16.090126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:16.092336 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:16.092796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:16.093265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:16.132282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1233/2450] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:23:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:23:16.187190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:16.190258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:16.205903 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1218/2451] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:16.497131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:16.500034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:16.596664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36736c2-5be3-46ba-8c46-1a24b27f858b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:16.601709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:16.602367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:16.633704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:16.633704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:16.633704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:16.633704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:16.633704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1234/2452] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:23:17.971111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:17.972637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:18.054449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bc109-f421-4016-b3f9-8adf81b92a7e service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:18.066730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:18.067475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:18.067758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:18.068034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:18.069906 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:18.083292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:18.085081 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:18.105390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1219/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:18.144510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:18.148000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:18.182236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088a832-bbad-4456-bc0d-57345ced4ead service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:18.189678 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:18.190708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:18.190993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:18.191278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] PATH_INFO: `/v3/users/8d154e96376a49918658b0c340c722e9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:18.192532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d154e96376a49918658b0c340c722e9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:18.194573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:18.570179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d154e96376a49918658b0c340c722e9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:18.570914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d154e96376a49918658b0c340c722e9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:18.590083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d154e96376a49918658b0c340c722e9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:18.784156 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d154e96376a49918658b0c340c722e9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:18.799002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4f3dd8d5-86f3-4276-a666-2693fd59639e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d154e96376a49918658b0c340c722e9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:18.805055 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1235/2454] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:18 2022] DELETE /identity/v3/users/8d154e96376a49918658b0c340c722e9 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:19.221981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:19.242664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554a41-1ee1-489a-b083-c5ef5b2ef4b6 service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:19.292872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:19.403746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:19.405156 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:19.410344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:19.411118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] PATH_INFO: `/v3/projects/84a2eee4447b4eadb79fbdbb874d4476` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:19.412876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a2eee4447b4eadb79fbdbb874d4476)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:19.416467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:19.655159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a2eee4447b4eadb79fbdbb874d4476', 'request_id': 'req-527a4346-9903-4728-b4b1-42be2c380f64'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:19.813125 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a2eee4447b4eadb79fbdbb874d4476', 'request_id': 'req-527a4346-9903-4728-b4b1-42be2c380f64'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:19.814546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a2eee4447b4eadb79fbdbb874d4476', 'request_id': 'req-527a4346-9903-4728-b4b1-42be2c380f64'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:19.826840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] The token cache is being invalidate because project 84a2eee4447b4eadb79fbdbb874d4476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:19.827138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:19.827976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-527a4346-9903-4728-b4b1-42be2c380f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:19.829752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1220/2455] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:19 2022] DELETE /identity/v3/projects/84a2eee4447b4eadb79fbdbb874d4476 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:19.834887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:19.835201 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:19.835786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:19.857538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:19.861071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:19.929449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1236/2456] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:19.938084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:19.938399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:19.939271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:19.961998 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:19.962970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:19.973919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1221/2457] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:19.978317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.000954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31ac0f-398c-4fb0-8132-411789d6e986 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.009511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.010131 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.010460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.010800 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.011472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:23:20.011785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.014461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.034473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1237/2458] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:23:20.039132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.051913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eba458-faa9-43dd-8fbc-b19019710545 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.060173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.060854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.061381 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.061823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.062746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.065149 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.222097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1222/2459] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:20 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:20.226269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.237811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7bed7-9d71-41c8-9b16-b2521124bd65 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.305930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.306651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.306983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.308400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.308400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.310248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.428366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1238/2460] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:20 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:20.429804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.442992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b9d2b-a026-426a-993e-b006383723ca admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.450250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.450901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.451748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.451748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.457594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.457594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.468874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1223/2461] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:20.470309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.480883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8a7e5-7747-4393-9b25-bf22c1e73337 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.487637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.488855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.489144 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.489430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.490119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.491902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885064fc-cdb5-4a23-9b35-bb68698bc14b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.505328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1239/2462] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:20.525611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.536446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77ee6-95ac-4b4f-bb73-970850756110 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.542097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.543108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.543541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.543986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] PATH_INFO: `/v3/projects/b703d830713d4bc0afb8622c93601b99/users/980fa47314ab45e68c4a43d485437fa9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.566273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b703d830713d4bc0afb8622c93601b99, user_id=980fa47314ab45e68c4a43d485437fa9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.569917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:20.698931 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1224/2463] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:20 2022] PUT /identity/v3/projects/b703d830713d4bc0afb8622c93601b99/users/980fa47314ab45e68c4a43d485437fa9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:20.714052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.715001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.715001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.754027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] MFA Rules not processed for user `980fa47314ab45e68c4a43d485437fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:20.755762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:20.847886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1240/2464] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:20.852629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:20.927315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828ee99-e372-4e64-8476-7a01f1029f26 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:20.932745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:20.933477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:20.933927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:20.934265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:20.935086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:20.937350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.092356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1225/2465] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:20 2022] POST /identity/v3/projects => generated 379 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:21.093934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:21.109059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689617-3341-4fc5-9ff3-1880007eb9a4 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:21.249147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:21.249855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.250077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.250361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.251959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:21.253683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.269663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:21.408570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1241/2466] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:21 2022] POST /identity/v3/users => generated 296 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:23:21.411690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:21.425025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f300140-4bdf-4b37-96ba-a3f1edb4d0f3 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:21.434010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:21.442303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.442303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.442303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.442303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:21.442303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37aa40-e8a4-432a-88c0-dbc5e5d5c3bf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.450112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1226/2467] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:21.455322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:21.466887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766c934-40d8-4d78-9f0f-0c2349eb5bdf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:21.477559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:21.477559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.477559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.477559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] PATH_INFO: `/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf/users/32fe9be9338a46fbb42b4dd3035bccc6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.508095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34d6b6dfa974d119ea77bb6291dcdcf, user_id=32fe9be9338a46fbb42b4dd3035bccc6, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:21.511315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.525643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.526019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.526298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.548335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] MFA Rules not processed for user `a9ebeec374454468aae1d2b5fde21648`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:21.550003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:21.621503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1227/2468] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:21.665651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:21.666172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1242/2469] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:21 2022] PUT /identity/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf/users/32fe9be9338a46fbb42b4dd3035bccc6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:21.690049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c3be7-1e2e-48ba-8510-86a17b827954 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:21.798433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:21.800181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.801458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.801742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.802567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:21.804552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1021520-8b41-4391-a4f1-639d6b3a27f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.828640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:21.829964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1228/2470] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:21.846695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257adb-3f36-4ce5-af54-abc05e6d0a4b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:21.874538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:21.875275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.875555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.876069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] PATH_INFO: `/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf/users/32fe9be9338a46fbb42b4dd3035bccc6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.883858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34d6b6dfa974d119ea77bb6291dcdcf, user_id=32fe9be9338a46fbb42b4dd3035bccc6, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:21.887069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:21.910274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1243/2471] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:21 2022] PUT /identity/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf/users/32fe9be9338a46fbb42b4dd3035bccc6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:21.921059 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:21.921411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:21.921708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:21.979538 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] MFA Rules not processed for user `32fe9be9338a46fbb42b4dd3035bccc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:21.981617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:22.120821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1229/2472] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:21 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:22.133503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:22.135823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:22.213842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b88c9-ab13-4f96-9fcc-f620769d2b00 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:22.217653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:22.218291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:22.218578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:22.218885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:22.232930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:22.234505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:22.246248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1244/2473] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:22.589475 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:22.603764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929d2b1-9e96-45b1-b301-c7977c2f495b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:22.614644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:22.615426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:22.615841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:22.616412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:22.686788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:22.688517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:22.703915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1230/2474] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:23.076986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:23.080552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:23.228398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480784-7b07-49dd-a6d6-4fb120d44a56 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:23.239543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:23.240267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:23.240940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:23.241221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:23.243645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:23.333091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:23.334976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:23.346396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1245/2475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:23.915102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:24.869170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:29.280242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:29.280242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:29.362493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169be9bc-db4c-4cd5-b1bd-6f64a63b972b service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:29.370486 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:29.371496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:29.372160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:29.372918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] PATH_INFO: `/v3/users/61318fafe2a1469c884b5290fe4ba366` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:29.374520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61318fafe2a1469c884b5290fe4ba366)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:29.377332 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:31.050257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61318fafe2a1469c884b5290fe4ba366'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:31.051066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61318fafe2a1469c884b5290fe4ba366'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:31.063206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61318fafe2a1469c884b5290fe4ba366'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:31.371757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61318fafe2a1469c884b5290fe4ba366'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:31.382200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61318fafe2a1469c884b5290fe4ba366'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:31.388495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1231/2476] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:29 2022] DELETE /identity/v3/users/61318fafe2a1469c884b5290fe4ba366 => generated 0 bytes in 2124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:31.968688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:31.981150 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a602094-84b2-43a0-9bd8-66027c7d3a5f service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:32.051617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:32.052272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:32.052561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:32.052921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] PATH_INFO: `/v3/projects/01817da1260c41b4abe6f02b11c66232` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:32.054185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01817da1260c41b4abe6f02b11c66232)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:32.056258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:32.332270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01817da1260c41b4abe6f02b11c66232', 'request_id': 'req-d80e77f8-e4e2-49c3-b286-ad3b9c124636'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:32.338207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01817da1260c41b4abe6f02b11c66232', 'request_id': 'req-d80e77f8-e4e2-49c3-b286-ad3b9c124636'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:32.531346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01817da1260c41b4abe6f02b11c66232', 'request_id': 'req-d80e77f8-e4e2-49c3-b286-ad3b9c124636'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:32.544619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] The token cache is being invalidate because project 01817da1260c41b4abe6f02b11c66232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:32.550546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:32.550994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:32.552745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1246/2477] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:31 2022] DELETE /identity/v3/projects/01817da1260c41b4abe6f02b11c66232 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:32.556981 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:32.558900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:32.582544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d7935-5f16-4b3f-9120-9089bfaffb8d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:32.645879 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:32.646575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:32.646909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:32.647189 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] PATH_INFO: `/v3/users/001ca23cef854ad1b8a234d80279bcfe` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:32.648417 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=001ca23cef854ad1b8a234d80279bcfe)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:32.650702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:32.697174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:33.015881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001ca23cef854ad1b8a234d80279bcfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:33.016952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001ca23cef854ad1b8a234d80279bcfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:33.035886 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001ca23cef854ad1b8a234d80279bcfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:33.137158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001ca23cef854ad1b8a234d80279bcfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:33.148798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-a9b2828e-a8e4-4f59-b90e-bc179a5f9fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001ca23cef854ad1b8a234d80279bcfe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:33.187531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1232/2478] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:32 2022] DELETE /identity/v3/users/001ca23cef854ad1b8a234d80279bcfe => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:33.464346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:33.748227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:33.750019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:33.769845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e77f8-e4e2-49c3-b286-ad3b9c124636 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:33.975988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:33.976639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:33.976924 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:33.977202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] PATH_INFO: `/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:33.978370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed8b47c2b8a495bb5fdb68cd8087d62)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:33.980360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:34.236701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed8b47c2b8a495bb5fdb68cd8087d62', 'request_id': 'req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:34.243672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed8b47c2b8a495bb5fdb68cd8087d62', 'request_id': 'req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:34.520023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed8b47c2b8a495bb5fdb68cd8087d62', 'request_id': 'req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:34.551351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] The token cache is being invalidate because project 5ed8b47c2b8a495bb5fdb68cd8087d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:34.551931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:34.554110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:34.556886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1247/2479] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:33 2022] DELETE /identity/v3/projects/5ed8b47c2b8a495bb5fdb68cd8087d62 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:34.758292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:37.280493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:37.710431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d489c-c9fc-409b-8679-93780491e969 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:37.710957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d489c-c9fc-409b-8679-93780491e969 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:37.716510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d489c-c9fc-409b-8679-93780491e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:37.732334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a37d489c-c9fc-409b-8679-93780491e969 None None] MFA Rules not processed for user `2f141efe3bb74587b5ba13f0cd5d89db`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:37.733544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a37d489c-c9fc-409b-8679-93780491e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:37.795571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1233/2480] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:38.497297 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:39.468859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:39.471198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:39.545620 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016076b-baaa-4ffc-85bd-5bdab8f4f1e7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:39.551419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:39.552068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:39.552348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:39.552624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] PATH_INFO: `/v3/users/980fa47314ab45e68c4a43d485437fa9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:39.553853 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=980fa47314ab45e68c4a43d485437fa9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:39.556013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:39.843875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980fa47314ab45e68c4a43d485437fa9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:39.844753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980fa47314ab45e68c4a43d485437fa9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:39.857565 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980fa47314ab45e68c4a43d485437fa9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:39.885820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:39.965884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980fa47314ab45e68c4a43d485437fa9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:39.974438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980fa47314ab45e68c4a43d485437fa9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:39.980574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1248/2481] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:39 2022] DELETE /identity/v3/users/980fa47314ab45e68c4a43d485437fa9 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:39.988364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:39.988364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:39.988364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:40.008208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:40.009411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:40.082175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1234/2482] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:40.093072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:40.095016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:40.156005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f120b4e-d82f-45c4-8f7c-2ba4f1663960 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:40.160334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:40.160954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:40.161254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:40.161553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:40.171139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:40.173136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:40.183160 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1249/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:40.868252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:40.876624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:40.901259 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea301293-6992-4560-98fa-f4c1ef0f57c9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:40.909625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:40.913761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:40.913761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:40.913761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] PATH_INFO: `/v3/projects/b703d830713d4bc0afb8622c93601b99` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:40.913761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b703d830713d4bc0afb8622c93601b99)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:40.918480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:41.176225 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b703d830713d4bc0afb8622c93601b99', 'request_id': 'req-65c459c6-ec28-4a58-9179-887a571bcafe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.371522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b703d830713d4bc0afb8622c93601b99', 'request_id': 'req-65c459c6-ec28-4a58-9179-887a571bcafe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.387028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b703d830713d4bc0afb8622c93601b99', 'request_id': 'req-65c459c6-ec28-4a58-9179-887a571bcafe'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.414588 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] The token cache is being invalidate because project b703d830713d4bc0afb8622c93601b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:41.415216 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.416006 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.419075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1235/2484] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:40 2022] DELETE /identity/v3/projects/b703d830713d4bc0afb8622c93601b99 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:41.424012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:41.436376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f2411-9fc8-4396-81cc-f3399712c5a6 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:41.505242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:41.505894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:41.506179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:41.507613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] PATH_INFO: `/v3/users/32fe9be9338a46fbb42b4dd3035bccc6` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:41.508251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fe9be9338a46fbb42b4dd3035bccc6)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:41.510203 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:41.761700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fe9be9338a46fbb42b4dd3035bccc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.762406 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fe9be9338a46fbb42b4dd3035bccc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:41.780909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fe9be9338a46fbb42b4dd3035bccc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:42.005750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fe9be9338a46fbb42b4dd3035bccc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:42.014531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6d551568-794e-4763-9c96-d57e4de33b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fe9be9338a46fbb42b4dd3035bccc6'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:42.021991 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1250/2485] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:41 2022] DELETE /identity/v3/users/32fe9be9338a46fbb42b4dd3035bccc6 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:42.741844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:42.741844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:42.751521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c459c6-ec28-4a58-9179-887a571bcafe admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:42.837621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:42.837621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:42.837621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:42.837621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] PATH_INFO: `/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:42.837950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34d6b6dfa974d119ea77bb6291dcdcf)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:42.838608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:43.119265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34d6b6dfa974d119ea77bb6291dcdcf', 'request_id': 'req-71b71c0f-2f49-4c23-9fea-525e7e8221f9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:43.684949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34d6b6dfa974d119ea77bb6291dcdcf', 'request_id': 'req-71b71c0f-2f49-4c23-9fea-525e7e8221f9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:43.694604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34d6b6dfa974d119ea77bb6291dcdcf', 'request_id': 'req-71b71c0f-2f49-4c23-9fea-525e7e8221f9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:43.708167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] The token cache is being invalidate because project b34d6b6dfa974d119ea77bb6291dcdcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:43.708614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:43.709422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71b71c0f-2f49-4c23-9fea-525e7e8221f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:43.711165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1236/2486] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:42 2022] DELETE /identity/v3/projects/b34d6b6dfa974d119ea77bb6291dcdcf => generated 0 bytes in 984 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:43.716647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:43.717129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:43.717287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:43.737702 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:43.739044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:43.802398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1251/2487] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:43.812710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:43.813800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:43.814026 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:43.834401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:43.853398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:43.853398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:43.859734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1237/2488] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:43.866784 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:43.879628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5234e7f-7bc2-49c7-b6de-b2f58643dc1a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:43.884701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:43.885329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:43.885609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:43.885882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:43.886950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:23:43.887252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:43.888950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:43.908088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1252/2489] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:23:43.913762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:43.926436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbddd0ed-3640-48a0-ac32-d12523c165dc None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:43.933832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:43.934521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:43.934845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:43.935179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:43.935858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:43.941357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:44.203675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1238/2490] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:43 2022] POST /identity/v3/projects => generated 373 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:44.204705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:44.220788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8662ca-bbe2-4bf2-a596-a981ed7b9cca admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:44.310095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:44.311323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.311863 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.313568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.318788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:44.318788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:44.318788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:44.459248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1253/2491] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:44 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:23:44.462911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:44.474738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89990a-1fea-43ab-81d7-e3601e3302dc admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:44.479867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:44.480575 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.481179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.481452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.482270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:44.484244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:44.498728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1239/2492] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:44.504700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:44.523498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcb2d-cb75-4448-9ed8-6ee8b9d696f8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:44.523498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:44.523498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.523498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.524018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.524660 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:44.526284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7da232-cb40-412c-bfd7-d02359ba22c3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:44.542584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:44.543388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1254/2493] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:44.556999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401c0e3-3486-4d57-a1b7-cdeeb293e19a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:44.562735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:44.564082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.564587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.564697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] PATH_INFO: `/v3/projects/47e3907d2ec647c5ab1de55d49251865/users/907534b491724cacb8a6cb4c74f66bd9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.581277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e3907d2ec647c5ab1de55d49251865, user_id=907534b491724cacb8a6cb4c74f66bd9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:44.584345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:44.778543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1240/2494] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:44 2022] PUT /identity/v3/projects/47e3907d2ec647c5ab1de55d49251865/users/907534b491724cacb8a6cb4c74f66bd9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:23:44.790064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.790667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.790787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.821234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] MFA Rules not processed for user `907534b491724cacb8a6cb4c74f66bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:44.822591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:44.896208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1255/2495] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:44.900662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:44.972674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf253ec-af99-418b-b631-de0e0685ea68 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:44.978229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:44.978603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:44.978915 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:44.979314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:44.980227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:44.982206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.233461 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1241/2496] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:44 2022] POST /identity/v3/projects => generated 373 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:45.235523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:45.249799 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89584f3-b330-4b37-b761-ca8e4609c488 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:45.355086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:45.355708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.355993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.356271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.356904 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:45.358886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.463846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1256/2497] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:45 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:45.467754 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:45.479088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99931aa-c827-4b82-953c-6cd45dce7121 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:45.484369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:45.484986 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.485269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.485728 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.486387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:45.488052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.502733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1242/2498] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:45.508422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:45.521231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebca15-8d8f-4e56-a315-f2d0b78ec82a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:45.534100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:45.534952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.535261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.535388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] PATH_INFO: `/v3/projects/257a6f3945e54dea951db425c9a71d4f/users/131a4adac80a4f73863152c3af81877a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.556866 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257a6f3945e54dea951db425c9a71d4f, user_id=131a4adac80a4f73863152c3af81877a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:45.559769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.783536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1257/2499] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:45 2022] PUT /identity/v3/projects/257a6f3945e54dea951db425c9a71d4f/users/131a4adac80a4f73863152c3af81877a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:45.790788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:45.802076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d724581-982e-4041-a208-f390bd44a7a2 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:45.867313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:45.867924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.868366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.868645 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.869290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:45.870948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0aa28-0cf3-4bf1-9ad2-3c1a84f6f21d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.884280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1243/2500] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:23:45.885697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:45.899830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb44d68-95d8-4a44-bf9a-e8c02a4807e6 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:45.904952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:45.905682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.906035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.906319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] PATH_INFO: `/v3/projects/257a6f3945e54dea951db425c9a71d4f/users/131a4adac80a4f73863152c3af81877a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.913568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257a6f3945e54dea951db425c9a71d4f, user_id=131a4adac80a4f73863152c3af81877a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:45.913568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:45.926134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1258/2501] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:45 2022] PUT /identity/v3/projects/257a6f3945e54dea951db425c9a71d4f/users/131a4adac80a4f73863152c3af81877a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:23:45.936776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:45.937284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:45.937537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:45.972715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] MFA Rules not processed for user `131a4adac80a4f73863152c3af81877a`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:45.974092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:46.054239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1244/2502] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:46.065599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:46.067514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:46.131827 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb06194-4860-473c-8eec-4d1db83acdc3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:46.135978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:46.136769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:46.138715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:46.138715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:46.151390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:46.153870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd033-2960-4525-926e-26e41bb1f9de service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:46.168610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1259/2503] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:48.217018 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:48.303199 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00bd0d0-47e7-4b87-83e1-ca85cf18a274 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:48.316985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:48.317643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:48.317957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:48.318238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] PATH_INFO: `/v3/users/907534b491724cacb8a6cb4c74f66bd9` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:48.319652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] RBAC: Authorizing `identity:delete_user(user_id=907534b491724cacb8a6cb4c74f66bd9)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:48.321591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:48.602021 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907534b491724cacb8a6cb4c74f66bd9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:48.602780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907534b491724cacb8a6cb4c74f66bd9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:48.615650 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907534b491724cacb8a6cb4c74f66bd9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:48.726850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907534b491724cacb8a6cb4c74f66bd9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:48.740242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907534b491724cacb8a6cb4c74f66bd9'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:48.767105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1245/2504] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:48 2022] DELETE /identity/v3/users/907534b491724cacb8a6cb4c74f66bd9 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:48.772266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:48.772758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:48.791492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:48.801846 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:48.803327 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:48.868374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1260/2505] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:48.879476 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:48.885147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:48.948521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f84ef9-bd13-4593-9695-405e7672365e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:48.951948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:48.952637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:48.952961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:48.953241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:48.964232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:48.965968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:48.978148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1246/2506] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:49.487415 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:49.489000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:49.503557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3db3c2-ab09-4e69-a0b0-5703260c2c50 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:49.508494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:49.509966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:49.510260 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:49.510537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] PATH_INFO: `/v3/projects/47e3907d2ec647c5ab1de55d49251865` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:49.511716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e3907d2ec647c5ab1de55d49251865)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:49.513840 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:49.749256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e3907d2ec647c5ab1de55d49251865', 'request_id': 'req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:49.756721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e3907d2ec647c5ab1de55d49251865', 'request_id': 'req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:49.874934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e3907d2ec647c5ab1de55d49251865', 'request_id': 'req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:49.887138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] The token cache is being invalidate because project 47e3907d2ec647c5ab1de55d49251865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:49.887433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:49.888102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:49.890115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1261/2507] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:49 2022] DELETE /identity/v3/projects/47e3907d2ec647c5ab1de55d49251865 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:49.895208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:49.905983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bc9f5-e543-4be9-a2f5-1e56942c748e service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:49.966294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:49.966992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:49.967269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:49.967540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] PATH_INFO: `/v3/users/131a4adac80a4f73863152c3af81877a` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:49.968668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=131a4adac80a4f73863152c3af81877a)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:49.971449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:50.097212 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:50.259556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131a4adac80a4f73863152c3af81877a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:50.260138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131a4adac80a4f73863152c3af81877a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:50.272630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131a4adac80a4f73863152c3af81877a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:50.417228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131a4adac80a4f73863152c3af81877a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:50.430747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-493afafb-aa04-4e42-b6d4-d78754852fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131a4adac80a4f73863152c3af81877a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:50.437619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1247/2508] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:23:49 2022] DELETE /identity/v3/users/131a4adac80a4f73863152c3af81877a => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:50.685767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:51.379336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:51.381547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:51.396914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78b5b7-259c-4d1f-8799-7b32c3117fd0 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:51.478307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:51.479249 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:51.479551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:51.480212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] PATH_INFO: `/v3/projects/257a6f3945e54dea951db425c9a71d4f` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:51.481584 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257a6f3945e54dea951db425c9a71d4f)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:51.483700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:51.748020 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257a6f3945e54dea951db425c9a71d4f', 'request_id': 'req-25e91b05-8de2-400d-ba78-2fd8d9a0d561'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:51.754238 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257a6f3945e54dea951db425c9a71d4f', 'request_id': 'req-25e91b05-8de2-400d-ba78-2fd8d9a0d561'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:51.914668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257a6f3945e54dea951db425c9a71d4f', 'request_id': 'req-25e91b05-8de2-400d-ba78-2fd8d9a0d561'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:51.927918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] The token cache is being invalidate because project 257a6f3945e54dea951db425c9a71d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:23:51.928214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:51.928870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-25e91b05-8de2-400d-ba78-2fd8d9a0d561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:23:51.930590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1262/2509] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:23:51 2022] DELETE /identity/v3/projects/257a6f3945e54dea951db425c9a71d4f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:23:51.935584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:51.936074 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:51.936349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:51.958143 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:51.959471 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:52.020843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1248/2510] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:52.028744 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.029031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.029332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.052938 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:52.054061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:52.063571 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1263/2511] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:52.067598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.077647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3be5e-e27b-436a-9c03-50c9f3db57e1 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.083746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.084377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.084688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.084964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.085680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:23:52.085943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.088010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.102665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1249/2512] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:23:52.106649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.116946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3307-956d-40b1-809d-d078bb4bd6f3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.126686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.126686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.126686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.126686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.127123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.127123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.320532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1264/2513] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:52 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:52.321460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.334031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189d7c-8a70-4f11-bdce-5dece3f284c7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.406581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.406581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.406581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.406581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.407159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.407159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.537762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1250/2514] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:52 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:52.543340 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.554721 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbe2e7-1e13-45cf-bc60-b77028736b91 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.559613 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.560326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.560694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.561209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.561940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.564647 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.576037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1265/2515] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:52.580867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.592347 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f57d4-9c0c-41c6-bb4f-ba72e79e3bb6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.597856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.598584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.601442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.601442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.601442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.601865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae1399-9fe7-4ae4-ab29-3bebb373dda6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.617283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1251/2516] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:52.618685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:52.631228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39607d67-38f1-4871-bec7-aed9c2b64423 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:52.636482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:52.637319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.641591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.641591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] PATH_INFO: `/v3/projects/893dbc6d3d3347c9995d4bd4f5dfbf48/users/919e332ae2644f90b67262eb692c7186/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.654922 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=893dbc6d3d3347c9995d4bd4f5dfbf48, user_id=919e332ae2644f90b67262eb692c7186, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:52.658001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:52.834969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1266/2517] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:52 2022] PUT /identity/v3/projects/893dbc6d3d3347c9995d4bd4f5dfbf48/users/919e332ae2644f90b67262eb692c7186/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:23:52.836862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:52.837151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:52.837428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:52.869848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] MFA Rules not processed for user `919e332ae2644f90b67262eb692c7186`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:52.871128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:52.944131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1252/2518] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:52.948971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.021834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e8261-7fb8-4771-80ac-bf750002d116 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.028549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.029186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.029475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.029752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.030386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.032967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.294422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1267/2519] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:23:52 2022] POST /identity/v3/projects => generated 377 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:53.299735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.313842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76187bc4-5d10-40e1-9d48-10450ace4812 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.378796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.379430 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.379694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.379962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.380599 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.382518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.584603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1253/2520] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:23:53 2022] POST /identity/v3/users => generated 295 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:23:53.588287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.611661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7409edf7-9864-4d86-b2d4-abf9886e33a2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.616797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.617576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.617950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.618323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.619134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.621037 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.634698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1268/2521] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:53.638974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.650321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1c2f-1886-40c9-8fe2-b3df4e98a00f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.656049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.657380 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.657690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.657971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] PATH_INFO: `/v3/projects/f5355f011a2648b085a554ca271ba006/users/2752c0d15714490f828a9220e67efebf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.674522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5355f011a2648b085a554ca271ba006, user_id=2752c0d15714490f828a9220e67efebf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.677557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.829765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1254/2522] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:53 2022] PUT /identity/v3/projects/f5355f011a2648b085a554ca271ba006/users/2752c0d15714490f828a9220e67efebf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:23:53.833873 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.844602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b674bd-503a-45cf-b79b-f29044f6074c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.909888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.910509 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.914828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.914828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.914828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.914828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c15b46-0097-4b04-97ac-c7d155e168c5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.924796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1269/2523] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:23:53.929577 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:53.940676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777708-9f5c-4633-a19a-c9803947d356 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:53.948106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:53.948106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.948106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.948106 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] PATH_INFO: `/v3/projects/f5355f011a2648b085a554ca271ba006/users/2752c0d15714490f828a9220e67efebf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:53.949767 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5355f011a2648b085a554ca271ba006, user_id=2752c0d15714490f828a9220e67efebf, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:53.952608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:53.964606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1255/2524] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:23:53 2022] PUT /identity/v3/projects/f5355f011a2648b085a554ca271ba006/users/2752c0d15714490f828a9220e67efebf/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:23:53.976479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:53.979388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:53.979388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:54.004688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] MFA Rules not processed for user `2752c0d15714490f828a9220e67efebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:23:54.005897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:54.080105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1270/2525] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:23:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:23:54.092868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:23:54.094820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:23:54.168116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f0f07-c3e4-4600-9bb1-0ecec6153c91 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:23:54.172011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:23:54.173400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:23:54.173899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:23:54.174179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:23:54.238177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:23:54.239708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:23:54.249776 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1256/2526] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:23:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:23:54.777135 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:23:57.720735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:00.102977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:00.743002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:01.258320 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:01.259811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:01.273200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e41a-a2e1-407d-ae2e-be1f99003f7c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:01.277861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:01.277861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:01.278123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:01.278567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:01.289440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:01.290926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:01.302207 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1271/2527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:02.846281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:03.175496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:03.177260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:03.235435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a1b27-3744-4396-85ec-990d02d5ec70 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:03.239449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:03.240068 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:03.242977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:03.242977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:03.242977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:03.307278 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:03.308727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:03.319157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1257/2528] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:03.868668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:04.006242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:04.693061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:04.694593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:04.708975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01f039-cc13-4704-bc8b-79be25fb0f21 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:04.715016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:04.715720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:04.716034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:04.716715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:04.718568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:04.793903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:04.794772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:04.805269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1272/2529] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:07.989258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:07.991377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:08.009670 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749988a4-aba6-48ea-9255-146120453346 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:08.018719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:08.019514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:08.020050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:08.020383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] PATH_INFO: `/v3/users/1d22ea3fb8fa4bd99537fb83279b2d24` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:08.021911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d22ea3fb8fa4bd99537fb83279b2d24)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:08.024683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:08.091760 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:08.831714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d22ea3fb8fa4bd99537fb83279b2d24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:08.832571 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d22ea3fb8fa4bd99537fb83279b2d24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:08.849403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d22ea3fb8fa4bd99537fb83279b2d24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:09.063005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d22ea3fb8fa4bd99537fb83279b2d24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:09.070468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2bf6abc3-b65e-4db1-b167-f3ddbabf634f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d22ea3fb8fa4bd99537fb83279b2d24'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:09.076591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1258/2530] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:07 2022] DELETE /identity/v3/users/1d22ea3fb8fa4bd99537fb83279b2d24 => generated 0 bytes in 1088 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:09.247083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:09.447538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:09.466710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0307ef-3a95-4d87-b596-db26a3fde0c6 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:09.525278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:09.526222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:09.526576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:09.526973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] PATH_INFO: `/v3/projects/aceecfa04cf941348d542c3e55569384` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:09.528335 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aceecfa04cf941348d542c3e55569384)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:09.530750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:09.842093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aceecfa04cf941348d542c3e55569384', 'request_id': 'req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:09.848505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aceecfa04cf941348d542c3e55569384', 'request_id': 'req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:10.101914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aceecfa04cf941348d542c3e55569384', 'request_id': 'req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:10.134192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] The token cache is being invalidate because project aceecfa04cf941348d542c3e55569384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:10.134192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:10.134192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-dc6aa4b7-08a9-4fc8-8a66-610c844a6297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:10.134192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1273/2531] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:09 2022] DELETE /identity/v3/projects/aceecfa04cf941348d542c3e55569384 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:10.137796 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.138298 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.138390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.160098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:10.161638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:10.225647 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1259/2532] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:10.233728 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.234018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.234380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.254687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:10.256106 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:10.264169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1274/2533] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:10.269250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.279288 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688192f5-72ab-4d3b-a8ac-a7f522980c68 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.284979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.295257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.305379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1260/2534] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:24:10.309841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.319325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb02e12-0115-4a93-b488-a66023bbcc3c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.324149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.324772 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.325053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.325330 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.325953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.328386 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.493956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1275/2535] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:10 2022] POST /identity/v3/projects => generated 357 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:10.498206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.508412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3835c1-6a63-489d-b718-041593dab843 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.568050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.568668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.568949 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.569232 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.569866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.571944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.696092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1261/2536] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:10 2022] POST /identity/v3/users => generated 279 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:10.696740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.712906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147aabb9-a428-4cdb-977a-17c1d2dce744 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.718946 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.719961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.720315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.721065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.721966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.726980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.749276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1276/2537] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:10.751398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.761574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd73ac-5d8c-4345-b536-8749c3e72af7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.766206 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.766869 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.767296 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.767578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.768222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.769811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda8da0-9c76-4ad7-84d7-dc859f160cb1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.780357 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1262/2538] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:10.787016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:10.798086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4062d3-ebe3-4ff2-bcbf-45640764f898 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:10.803101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:10.803760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.804048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.804331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] PATH_INFO: `/v3/projects/d9e1377adbf947af9019c9608b11f231/users/07f19d87db8f4803a9babe3d50eb5157/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:10.819769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e1377adbf947af9019c9608b11f231, user_id=07f19d87db8f4803a9babe3d50eb5157, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:10.822507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:10.978031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1277/2539] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:10 2022] PUT /identity/v3/projects/d9e1377adbf947af9019c9608b11f231/users/07f19d87db8f4803a9babe3d50eb5157/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:10.989365 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:10.989676 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:10.989935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.041766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] MFA Rules not processed for user `07f19d87db8f4803a9babe3d50eb5157`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:11.053924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:11.116144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1263/2540] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:11.123312 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.193206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795127b0-aa11-42ba-a2f3-12845ae5aff2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.198239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.198887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.199192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.199512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.200174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.202212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.383889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1278/2541] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:11 2022] POST /identity/v3/projects => generated 357 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:11.388277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.399629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eca21-c0ef-48a3-b632-ae9cbe8b49cd None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.456783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.457396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.457680 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.457955 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.458587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.460540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.643837 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1264/2542] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:11 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:11.644995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.655095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ecebc0-89d4-41c8-bdd9-70a6053018a5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.660805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.661423 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.661723 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.662094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.663004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.664664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.677002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1279/2543] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:11.681257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.691648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200447f9-a060-4928-918c-39cb534dbb53 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.696730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.697392 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.697665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.697943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] PATH_INFO: `/v3/projects/64c48d4e90e94510a172e8a35e529713/users/59611cc40b184a6eae765ecb9498ec2a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.713992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c48d4e90e94510a172e8a35e529713, user_id=59611cc40b184a6eae765ecb9498ec2a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.724163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.840823 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1265/2544] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:11 2022] PUT /identity/v3/projects/64c48d4e90e94510a172e8a35e529713/users/59611cc40b184a6eae765ecb9498ec2a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:11.845823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.856125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708af5-d109-4cb8-9009-5e1dfe01f3b9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.919036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.919659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.919960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.920250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.920899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.922729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e0552-5427-4706-88ff-473e0065cef5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.933072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1280/2545] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:11.937956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:11.948414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eea68-102f-4c1a-84dc-d93003219f74 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:11.953946 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:11.954714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.955005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.955315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] PATH_INFO: `/v3/projects/64c48d4e90e94510a172e8a35e529713/users/59611cc40b184a6eae765ecb9498ec2a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:11.957376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c48d4e90e94510a172e8a35e529713, user_id=59611cc40b184a6eae765ecb9498ec2a, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:11.960087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:11.972195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1266/2546] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:11 2022] PUT /identity/v3/projects/64c48d4e90e94510a172e8a35e529713/users/59611cc40b184a6eae765ecb9498ec2a/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:11.982303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:11.982600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:11.983003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:12.012925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] MFA Rules not processed for user `59611cc40b184a6eae765ecb9498ec2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:12.014184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:12.086307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1281/2547] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:12.097924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:12.099750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:12.121474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:12.163658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f928cb-5bd9-4f4a-8c7c-a330b0ecda06 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:12.167437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:12.168040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:12.168321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:12.168591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:12.182369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:12.183863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:12.194707 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1267/2548] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:12.315012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:12.459527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:12.470376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73b165-3d90-4b48-aa9c-a73d7a4e5ecd None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:12.476640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:12.477225 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:12.477557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:12.478076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:12.549304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:12.551045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c0de8-63a4-4f5a-97d6-9bec194b13d4 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:12.563035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1282/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:12.622474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:14.474202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:15.886017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:16.697796 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:17.645230 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:18.851564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:19.107965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:20.154004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:20.257087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:20.966042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:21.748455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:22.919750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:24.085096 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:25.326030 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:25.440719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:26.131396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:26.708383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:27.549142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:27.800995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:29.198416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:31.114318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:32.876627 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:32.878173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:32.982540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:33.033013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51930437-ee43-4373-9a36-591e3bd625a5 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:33.038828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:33.039641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:33.039965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:33.040281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] PATH_INFO: `/v3/users/919e332ae2644f90b67262eb692c7186` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:33.041737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=919e332ae2644f90b67262eb692c7186)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:33.043832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:33.288120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919e332ae2644f90b67262eb692c7186'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:33.288965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919e332ae2644f90b67262eb692c7186'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:33.307375 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919e332ae2644f90b67262eb692c7186'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:33.381970 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919e332ae2644f90b67262eb692c7186'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:33.390757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919e332ae2644f90b67262eb692c7186'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:33.399061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1268/2550] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:32 2022] DELETE /identity/v3/users/919e332ae2644f90b67262eb692c7186 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:24:33.403641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:33.404198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:33.404198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:33.436117 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:33.437460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:33.508602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1283/2551] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:33.519454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:33.521024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:33.603033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341738e-257e-4fde-8307-7e2b9ae7bdb1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:33.606851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:33.607593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:33.607747 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:33.608024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:33.619327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:33.621179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:33.632167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1269/2552] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:34.133836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:34.135430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:34.149649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844a65b-97cd-4eb0-9595-91736a6b1565 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:34.158346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:34.158346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:34.158346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:34.158346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] PATH_INFO: `/v3/projects/893dbc6d3d3347c9995d4bd4f5dfbf48` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:34.158821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=893dbc6d3d3347c9995d4bd4f5dfbf48)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:34.160370 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:34.287045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '893dbc6d3d3347c9995d4bd4f5dfbf48', 'request_id': 'req-bfb78be7-8a26-4079-96c4-024caadd39c5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.295472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:34.297748 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '893dbc6d3d3347c9995d4bd4f5dfbf48', 'request_id': 'req-bfb78be7-8a26-4079-96c4-024caadd39c5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.365487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '893dbc6d3d3347c9995d4bd4f5dfbf48', 'request_id': 'req-bfb78be7-8a26-4079-96c4-024caadd39c5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.378298 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] The token cache is being invalidate because project 893dbc6d3d3347c9995d4bd4f5dfbf48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:34.378632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.379319 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.380794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1284/2553] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:34 2022] DELETE /identity/v3/projects/893dbc6d3d3347c9995d4bd4f5dfbf48 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:34.384822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:34.395157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281ae4e-4de4-485e-bc36-455b2918cdac service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:34.416523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:34.457700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:34.459845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:34.459845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:34.459845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] PATH_INFO: `/v3/users/2752c0d15714490f828a9220e67efebf` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:34.460247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2752c0d15714490f828a9220e67efebf)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:34.462185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:34.706833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2752c0d15714490f828a9220e67efebf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.707662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2752c0d15714490f828a9220e67efebf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.724333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2752c0d15714490f828a9220e67efebf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.829154 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2752c0d15714490f828a9220e67efebf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.839939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d61c3ae5-406e-4efd-b711-b04d3d6e9723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2752c0d15714490f828a9220e67efebf'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:34.850856 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1270/2554] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:34 2022] DELETE /identity/v3/users/2752c0d15714490f828a9220e67efebf => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:35.366661 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:35.370646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:35.385595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb78be7-8a26-4079-96c4-024caadd39c5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:35.449805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:35.450473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:35.450930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:35.451230 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] PATH_INFO: `/v3/projects/f5355f011a2648b085a554ca271ba006` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:35.452375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5355f011a2648b085a554ca271ba006)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:35.455473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:35.594334 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5355f011a2648b085a554ca271ba006', 'request_id': 'req-3298e73b-c386-458d-a3da-29ea753f28d8'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:35.603164 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5355f011a2648b085a554ca271ba006', 'request_id': 'req-3298e73b-c386-458d-a3da-29ea753f28d8'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:35.696766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5355f011a2648b085a554ca271ba006', 'request_id': 'req-3298e73b-c386-458d-a3da-29ea753f28d8'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:35.712594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] The token cache is being invalidate because project f5355f011a2648b085a554ca271ba006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:35.712894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:35.713644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-3298e73b-c386-458d-a3da-29ea753f28d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:35.715283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1285/2555] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:35 2022] DELETE /identity/v3/projects/f5355f011a2648b085a554ca271ba006 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:35.721436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:35.722355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:35.722662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:35.781013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:35.782652 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:35.854345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1271/2556] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:35.863419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:35.863883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:35.864348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:35.889774 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:35.892174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:35.901551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1286/2557] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:35.906755 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:35.933282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adede2-71ac-4e3e-86f6-6f11747b0e5e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:35.933282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:35.933282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:35.933282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:35.933706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:35.933706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:35.933883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:35.935452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:35.948852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1272/2558] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:24:35.954894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:35.965590 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b76ef-a1d2-418a-b9ee-3a4a4805936b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:35.972265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:35.972265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:35.972942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:35.972942 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:35.973204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:35.975247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.071535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1287/2559] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:35 2022] POST /identity/v3/projects => generated 401 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:36.084130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.089521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425fafe-864d-4808-a8ef-6fb540cf28f1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:36.100615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:36.159786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:36.160419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.160699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.160972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.161629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:36.163702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.308598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1273/2560] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:36 2022] POST /identity/v3/users => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:36.313405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.328088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d727be5-87a3-4f35-a3a5-4ee5f9dd042e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:36.334829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:36.336524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.336524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.336524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.336813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:36.338191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.349278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1288/2561] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:36.353542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.364316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9461e-b343-4eb3-8fdd-c457302a523c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:36.368893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:36.369485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.369764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.370035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.370723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:36.372369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084abc54-1808-4a70-bfba-aa8522cca15a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.383604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1274/2562] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:36.387751 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.398785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc0404-4959-46fc-ad55-d62cf2935c48 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:36.403400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:36.404046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.404328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.404603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] PATH_INFO: `/v3/projects/32aafc4712db45f9897e753599950d41/users/d83240a0b42044b7be1bae2b87eb08c5/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.420181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32aafc4712db45f9897e753599950d41, user_id=d83240a0b42044b7be1bae2b87eb08c5, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:36.423163 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.566254 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1289/2563] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:36 2022] PUT /identity/v3/projects/32aafc4712db45f9897e753599950d41/users/d83240a0b42044b7be1bae2b87eb08c5/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:36.578289 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.578551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.578855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.609340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] MFA Rules not processed for user `d83240a0b42044b7be1bae2b87eb08c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:36.610633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:36.707092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1275/2564] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:36 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:36.711533 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.728051 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:36.794175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5cc9-3599-42f3-8b41-b021d375017d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:36.800765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:36.801353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:36.801821 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:36.802280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:36.803364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:36.807026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:36.933114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1290/2565] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:36 2022] POST /identity/v3/projects => generated 397 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:36.937434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:36.948804 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86925d-d26a-46f1-824b-9efebb96b130 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.010035 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.010681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.010965 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.011231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.011954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.013820 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.164742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1276/2566] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:36 2022] POST /identity/v3/users => generated 305 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:37.168546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.179652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0b03-dc3e-48bb-90bc-8a9a3358039a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.184385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.184988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.185316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.185695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.186345 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.188579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28215bc-94d0-4bf6-9e06-e905d046fa5e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.200472 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1291/2567] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:37.205526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.216088 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c9b4b-735a-4705-874b-1aa98ae713be admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.221825 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.222522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.222836 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.223115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] PATH_INFO: `/v3/projects/b475733514674b3f94d4d53012432ad6/users/97e68cc42db14b8fb48abfa420c1ad90/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.241057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b475733514674b3f94d4d53012432ad6, user_id=97e68cc42db14b8fb48abfa420c1ad90, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.244325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.262153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.262459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.262716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.301296 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:37.304615 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:37.345644 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1277/2568] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:37 2022] PUT /identity/v3/projects/b475733514674b3f94d4d53012432ad6/users/97e68cc42db14b8fb48abfa420c1ad90/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:24:37.350466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.391957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1292/2569] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:24:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:37.442601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95315d-d18e-4bcf-9463-966789404337 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.447514 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.448121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.448401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.448675 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.449752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.451451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc566a7d-4fd6-45fd-bfaa-25856de783a9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.462919 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1278/2570] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:37.467753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.480182 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee038b12-b2f2-47ba-a5a4-ca43f7322279 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.485161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.486004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.486294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.486575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] PATH_INFO: `/v3/projects/b475733514674b3f94d4d53012432ad6/users/97e68cc42db14b8fb48abfa420c1ad90/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.488619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b475733514674b3f94d4d53012432ad6, user_id=97e68cc42db14b8fb48abfa420c1ad90, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.491482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.507614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1293/2571] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:37 2022] PUT /identity/v3/projects/b475733514674b3f94d4d53012432ad6/users/97e68cc42db14b8fb48abfa420c1ad90/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:37.518958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.519244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.519521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.549689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] MFA Rules not processed for user `97e68cc42db14b8fb48abfa420c1ad90`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:37.551235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:37.587861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.589882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:37.693273 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518d3f9-493a-40bd-a5f1-7ac34d0d7e69 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.698478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.699153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.699446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.699722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.762522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1279/2572] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:37 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:37.764420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.766715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.772273 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:37.774090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:37.779812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1294/2573] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:24:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 06:24:37.868039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f0026-5a83-4dc7-9d6f-d853934d6674 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:37.874653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:37.875482 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:37.875774 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:37.876052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:37.896295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:37.898067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:37.910941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1280/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:37 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:38.234026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:38.245806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:38.252579 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65f7d1-7930-439d-91f1-30653ae6610e service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:38.257178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:38.257914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:38.258302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:38.258714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:38.325087 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:38.326513 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:38.338814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1295/2575] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:39.267874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:39.609355 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:39.615328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:39.627266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:39.644227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:39.645792 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:39.725979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28adb9-04bf-42b9-9a9b-2ff3de3f942c service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:39.735743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:39.736495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:39.736779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:39.737056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] PATH_INFO: `/v3/users/07f19d87db8f4803a9babe3d50eb5157` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:39.738544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f19d87db8f4803a9babe3d50eb5157)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:39.743641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:39.831889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97fcc-e679-4d30-a91b-753dbb6bd781 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:39.837523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:39.838154 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:39.838440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:39.838753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:39.841194 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:39.855611 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:39.859162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e74f65-90bd-496d-be7d-fe3be5c86f4c service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:39.899305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1296/2576] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:24:40.057523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f19d87db8f4803a9babe3d50eb5157'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:40.058190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f19d87db8f4803a9babe3d50eb5157'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:40.082324 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f19d87db8f4803a9babe3d50eb5157'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:40.200858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f19d87db8f4803a9babe3d50eb5157'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:40.211504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f19d87db8f4803a9babe3d50eb5157'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:40.224694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1281/2577] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:39 2022] DELETE /identity/v3/users/07f19d87db8f4803a9babe3d50eb5157 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:40.226119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:40.226505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:40.226870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:40.250027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:40.251379 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:40.373267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1297/2578] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:40.384622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:40.387119 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:40.490982 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbdab7-5dbc-490b-bd9f-f6f856e6759e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:40.498384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:40.499963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:40.500479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:40.500686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:40.520014 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:40.521614 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:40.537442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1282/2579] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:40.668443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:41.974358 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:41.976504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:41.993132 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73796081-e0cb-4448-8906-f0fa3b94dc7f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:42.013562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:42.015530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:42.016882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:42.018009 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] PATH_INFO: `/v3/projects/d9e1377adbf947af9019c9608b11f231` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:42.020569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e1377adbf947af9019c9608b11f231)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:42.022903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:42.282023 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e1377adbf947af9019c9608b11f231', 'request_id': 'req-085a74e9-9c70-4635-b0f5-d6219b70b6f0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.289838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e1377adbf947af9019c9608b11f231', 'request_id': 'req-085a74e9-9c70-4635-b0f5-d6219b70b6f0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.416635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e1377adbf947af9019c9608b11f231', 'request_id': 'req-085a74e9-9c70-4635-b0f5-d6219b70b6f0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.437178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] The token cache is being invalidate because project d9e1377adbf947af9019c9608b11f231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:42.437500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.438283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.440167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1298/2580] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:41 2022] DELETE /identity/v3/projects/d9e1377adbf947af9019c9608b11f231 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:42.444173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:42.456785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a323343-5d4d-4455-851a-243a067c4204 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:42.492621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:42.537215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:42.538201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:42.538562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:42.540694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] PATH_INFO: `/v3/users/59611cc40b184a6eae765ecb9498ec2a` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:42.540694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59611cc40b184a6eae765ecb9498ec2a)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:42.542502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:42.829870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59611cc40b184a6eae765ecb9498ec2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.833537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59611cc40b184a6eae765ecb9498ec2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.848938 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59611cc40b184a6eae765ecb9498ec2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.959633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59611cc40b184a6eae765ecb9498ec2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.967634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86a83624-1ad2-4555-973a-0a3a9953a536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59611cc40b184a6eae765ecb9498ec2a'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:42.976624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1283/2581] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:42 2022] DELETE /identity/v3/users/59611cc40b184a6eae765ecb9498ec2a => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:43.471214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:43.472845 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:43.543888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a74e9-9c70-4635-b0f5-d6219b70b6f0 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:43.718014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:43.718703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:43.718998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:43.719277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] PATH_INFO: `/v3/projects/64c48d4e90e94510a172e8a35e529713` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:43.720464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c48d4e90e94510a172e8a35e529713)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:43.722931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:43.865240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:43.872989 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c48d4e90e94510a172e8a35e529713', 'request_id': 'req-868056e6-c86c-43c9-9380-7be93424c5ba'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:43.888732 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c48d4e90e94510a172e8a35e529713', 'request_id': 'req-868056e6-c86c-43c9-9380-7be93424c5ba'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:43.993482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c48d4e90e94510a172e8a35e529713', 'request_id': 'req-868056e6-c86c-43c9-9380-7be93424c5ba'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:44.007507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] The token cache is being invalidate because project 64c48d4e90e94510a172e8a35e529713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:44.007905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:44.008478 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-868056e6-c86c-43c9-9380-7be93424c5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:44.013408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1299/2582] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:43 2022] DELETE /identity/v3/projects/64c48d4e90e94510a172e8a35e529713 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:24:44.015978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.016507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.016719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.040410 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:44.041628 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:44.114668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1284/2583] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:44.123439 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.123966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.124844 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.148638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:44.148638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:44.157493 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1300/2584] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:44.162188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.173913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec09e3-4ce1-420d-92f6-4652ed2a675f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.179551 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.180244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.180598 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.180978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.181738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:44.182117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.183948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:44.200093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1285/2585] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:24:44.204777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.216548 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803dee-e1dc-4d84-aaf0-8f05510732db None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.223060 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.223901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.224476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.224932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.225767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.228124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:44.373578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1301/2586] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:44 2022] POST /identity/v3/projects => generated 363 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:44.380228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.394157 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d14634-bc34-4ad0-98a4-3f9c06d9b335 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.433240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:44.466715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.467464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.467749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.468076 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.469080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.471303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:44.630491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1286/2587] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:44 2022] POST /identity/v3/users => generated 282 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:44.634944 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.646676 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f3977-3c2f-4025-9169-8350f85ca386 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.651961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.652798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.653093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.653375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.654054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.657159 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:44.669459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1302/2588] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:44.675899 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.688080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16993a-11b9-4dcb-8ff3-e462a87a51e5 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.693165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.693816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.694098 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.694377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.695072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.696812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ba17d-183f-46c8-b27c-dad2212fb4cf admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:44.708830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1287/2589] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:44.713435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:44.725550 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136941e-038e-4eab-84d2-1adf9648c16f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:44.730553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:44.732027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:44.732336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:44.732479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] PATH_INFO: `/v3/projects/8f504e32d97a4e0aa98f197dd4d35cfa/users/f7e198cb3f2a4b0390e25d1678b1ce7b/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:44.750204 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f504e32d97a4e0aa98f197dd4d35cfa, user_id=f7e198cb3f2a4b0390e25d1678b1ce7b, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:44.753206 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.169962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1303/2590] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:44 2022] PUT /identity/v3/projects/8f504e32d97a4e0aa98f197dd4d35cfa/users/f7e198cb3f2a4b0390e25d1678b1ce7b/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:45.181387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.181909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.181909 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.212545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] MFA Rules not processed for user `f7e198cb3f2a4b0390e25d1678b1ce7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:45.213782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:45.282866 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1288/2591] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:45.287495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.351698 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945d340-e6a8-487b-b6d6-07ed05b9fe6d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:45.356213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:45.357079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.357651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.358065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.358645 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:45.360958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.403046 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.409300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:45.420425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf3a2c-5f5b-4ef7-9be2-c0a7b8cd2cd8 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:45.425722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:45.429496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.429496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.429496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] PATH_INFO: `/v3/users/d83240a0b42044b7be1bae2b87eb08c5` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.429496 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d83240a0b42044b7be1bae2b87eb08c5)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:45.430176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.503945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1304/2592] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:45 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:45.504737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.573969 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bb9f7-5f47-446c-87d3-0c4fd9c50181 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:45.579092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:45.579706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.579988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.580261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.580887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:45.582919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.701083 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1305/2593] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:45 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:45.706519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.724559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9632c-e3d7-4a26-966d-5e771c2cd899 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:45.732193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:45.733176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.734414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.734921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.736935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:45.742310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.744879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:45.760139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1306/2594] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:45.761498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.762110 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd83240a0b42044b7be1bae2b87eb08c5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:45.763016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd83240a0b42044b7be1bae2b87eb08c5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:45.776228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24975f-d219-4a90-b19a-58d0c8e6c472 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:45.780226 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd83240a0b42044b7be1bae2b87eb08c5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:45.785033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:45.791251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.791251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.791251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] PATH_INFO: `/v3/projects/815e3cf08c8d468688c95b7b28021634/users/f6069f6622634b048168804a7cc0f5b7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:45.803649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=815e3cf08c8d468688c95b7b28021634, user_id=f6069f6622634b048168804a7cc0f5b7, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:45.806474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:45.953490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd83240a0b42044b7be1bae2b87eb08c5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:45.959995 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1307/2595] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:45 2022] PUT /identity/v3/projects/815e3cf08c8d468688c95b7b28021634/users/f6069f6622634b048168804a7cc0f5b7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:45.968710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-66582f65-a51b-4084-bf7c-eaab5592dbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd83240a0b42044b7be1bae2b87eb08c5'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:45.968870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:45.968870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:45.971450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1289/2596] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:45 2022] DELETE /identity/v3/users/d83240a0b42044b7be1bae2b87eb08c5 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:24:45.975762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:45.976304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:45.976745 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.004065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:46.005936 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:46.042171 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983e22a-7cfb-4e08-8528-601e604f8dbe admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:46.122190 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1290/2597] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:46.128217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:46.128875 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.129715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.130000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.131314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:46.133435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:46.135912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:46.138830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:46.151600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1308/2598] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:46.154544 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:46.166928 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa1c55-1e61-4621-ac7e-15d154815f21 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:46.175831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:46.176771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.177067 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.177344 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] PATH_INFO: `/v3/projects/815e3cf08c8d468688c95b7b28021634/users/f6069f6622634b048168804a7cc0f5b7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.179878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=815e3cf08c8d468688c95b7b28021634, user_id=f6069f6622634b048168804a7cc0f5b7, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:46.183414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d17b7-7b09-49ae-88b8-17fd4f0ffb0a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:46.197824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1309/2599] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:46 2022] PUT /identity/v3/projects/815e3cf08c8d468688c95b7b28021634/users/f6069f6622634b048168804a7cc0f5b7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:46.210767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.211398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.211511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.253118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] MFA Rules not processed for user `f6069f6622634b048168804a7cc0f5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:46.254411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:46.289311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5212f-b7f2-4d88-9ac7-864389dc58c1 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:46.352077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1310/2600] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:46.367536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:46.371621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:46.388412 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:46.390439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:46.391387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.391904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.392557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.455510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9959213-da3a-4efe-9efc-e1ae15f77836 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:46.459664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:46.460394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.460697 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.461048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.480121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:46.481730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:46.496417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1311/2601] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:46.510334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:46.511943 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:46.541563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1291/2602] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:24:46.808325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:46.818906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4b08c-b42c-4a21-b5f0-e278d450e833 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:46.823165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:46.823918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:46.824304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:46.827436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:46.838549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:46.930028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:46.931671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:46.945994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1312/2603] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:47.054441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:47.056150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:47.071842 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6703d-c45c-4d88-9b4d-c57fddd77e6d service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:47.084052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:47.085124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:47.085916 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:47.086426 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] PATH_INFO: `/v3/projects/32aafc4712db45f9897e753599950d41` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:47.086903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:47.088944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32aafc4712db45f9897e753599950d41)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:47.091384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:47.292791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32aafc4712db45f9897e753599950d41', 'request_id': 'req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.372901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32aafc4712db45f9897e753599950d41', 'request_id': 'req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.388924 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32aafc4712db45f9897e753599950d41', 'request_id': 'req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.405448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] The token cache is being invalidate because project 32aafc4712db45f9897e753599950d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:47.405739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.406548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.411292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1292/2604] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:47 2022] DELETE /identity/v3/projects/32aafc4712db45f9897e753599950d41 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:47.415424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:47.433179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34f885-72a4-4288-a476-99e5cdab648d service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:47.518487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:47.519169 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:47.519455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:47.519733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] PATH_INFO: `/v3/users/97e68cc42db14b8fb48abfa420c1ad90` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:47.520975 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e68cc42db14b8fb48abfa420c1ad90)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:47.523127 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:47.605511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:47.760915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e68cc42db14b8fb48abfa420c1ad90'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.761617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e68cc42db14b8fb48abfa420c1ad90'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.777317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e68cc42db14b8fb48abfa420c1ad90'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.879882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:47.901964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e68cc42db14b8fb48abfa420c1ad90'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.915518 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-782cc9ef-1bd3-40ca-bf3f-36e28b861670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e68cc42db14b8fb48abfa420c1ad90'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:47.925879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1313/2605] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:47 2022] DELETE /identity/v3/users/97e68cc42db14b8fb48abfa420c1ad90 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:48.539092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:48.541223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:48.601040 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7f0bc-db15-4d22-83f8-ecc60bd0c5c7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:48.635293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:48.635959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:48.636619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:48.636619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] PATH_INFO: `/v3/projects/b475733514674b3f94d4d53012432ad6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:48.638038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b475733514674b3f94d4d53012432ad6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:48.640466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:48.912749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b475733514674b3f94d4d53012432ad6', 'request_id': 'req-85f118d4-8724-42af-ba64-d8ad7e0b3775'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:49.148564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b475733514674b3f94d4d53012432ad6', 'request_id': 'req-85f118d4-8724-42af-ba64-d8ad7e0b3775'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:49.156181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b475733514674b3f94d4d53012432ad6', 'request_id': 'req-85f118d4-8724-42af-ba64-d8ad7e0b3775'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:49.170481 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] The token cache is being invalidate because project b475733514674b3f94d4d53012432ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:49.171340 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:49.178805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-85f118d4-8724-42af-ba64-d8ad7e0b3775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:49.178805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1293/2606] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:48 2022] DELETE /identity/v3/projects/b475733514674b3f94d4d53012432ad6 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:49.180241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.180692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.180787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.203263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:49.204512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:49.270776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1314/2607] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:49.279020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.279303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.279581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.302510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:49.303772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:49.312504 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1294/2608] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:49.316912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.327455 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398e5e9-cf0a-4d7b-b62c-573acb0f0723 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.332104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.332711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.332998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.333276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.333927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:49.334237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.336027 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:49.348428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1315/2609] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:24:49.352972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.363345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdaa6b-d7f7-42db-99f2-b5e01cc8a8b5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.369634 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.370677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.371173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.371662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.372764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.375094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:49.380055 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:49.609726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1295/2610] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:49 2022] POST /identity/v3/projects => generated 409 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:49.611065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.622277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deb756-0974-4647-8524-bee0a11d5307 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.689808 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.695897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.695897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.695897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.695897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.695897 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:49.709130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:49.835549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1316/2611] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:49 2022] POST /identity/v3/users => generated 305 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:49.836167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.848928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2905f8a-387f-469b-a9f4-6b2a5945e9cd admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.854329 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.855142 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.855565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.856090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.856865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.858862 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:49.870665 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1296/2612] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:49.875292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.887811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaba30e-874d-4837-84f8-c09b263a899c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.920905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.920905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.920905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.920905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.921355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.921355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596e771-f4ef-4c29-ba07-73effbf62e21 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:49.929655 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1317/2613] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:49.933708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:49.945956 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770eb66-66b9-456a-b6e8-e6813aae85d9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:49.951092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:49.951761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:49.952814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:49.953280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] PATH_INFO: `/v3/projects/9a46d2e7309e4b6883b07521a29d8b33/users/b7ce0d49e0e14d1fbc103ae528c05e60/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:49.973671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a46d2e7309e4b6883b07521a29d8b33, user_id=b7ce0d49e0e14d1fbc103ae528c05e60, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:49.976908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:50.069229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1297/2614] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:49 2022] PUT /identity/v3/projects/9a46d2e7309e4b6883b07521a29d8b33/users/b7ce0d49e0e14d1fbc103ae528c05e60/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:50.143398 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:50.143683 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:50.143955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:50.176364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] MFA Rules not processed for user `b7ce0d49e0e14d1fbc103ae528c05e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:50.177553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:50.256241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1318/2615] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:50 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:50.260746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:50.344541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaed024-0043-424d-9082-74c66c34f041 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:50.349522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:50.349975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:50.350255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:50.350530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:50.351195 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:50.353159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:50.486992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1298/2616] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:50 2022] POST /identity/v3/projects => generated 409 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:50.491129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:50.502174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8439af9-0466-4b8f-8dac-670d16346007 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:50.564977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:50.577686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:50.577686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:50.577686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:50.577686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:50.577686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:50.759309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1319/2617] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:50 2022] POST /identity/v3/users => generated 311 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:50.763579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:50.776314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf8bf-780e-406e-83ea-fc7c7dfe51d6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:50.780921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:50.781522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:50.781802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:50.782073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:50.782752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:50.784341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:50.796343 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1299/2618] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:50.801621 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:50.812236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a99e-62d3-4e06-976b-d6c38d747f4a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:50.816906 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:50.817556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:50.817841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:50.818122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] PATH_INFO: `/v3/projects/7682e0952d07486086a06ab01572bc6e/users/c4e915efebdf4018bec8e7fc9c010440/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:50.833976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7682e0952d07486086a06ab01572bc6e, user_id=c4e915efebdf4018bec8e7fc9c010440, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:50.842139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:50.976524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1320/2619] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:50 2022] PUT /identity/v3/projects/7682e0952d07486086a06ab01572bc6e/users/c4e915efebdf4018bec8e7fc9c010440/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:50.980537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:50.990543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f002ee1-3044-4d15-a420-48b3e615c95f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:51.047524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:51.048138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.060651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.060651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.060651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:51.060651 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c30c4-c8aa-49f7-816f-f07a7569ec66 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:51.061957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1300/2620] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:51.066071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:51.078233 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1059f-2984-429d-9324-8d2a28c20d65 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:51.084228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:51.084899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.085870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.086185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] PATH_INFO: `/v3/projects/7682e0952d07486086a06ab01572bc6e/users/c4e915efebdf4018bec8e7fc9c010440/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.088670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7682e0952d07486086a06ab01572bc6e, user_id=c4e915efebdf4018bec8e7fc9c010440, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:51.091408 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:51.102752 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1321/2621] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:51 2022] PUT /identity/v3/projects/7682e0952d07486086a06ab01572bc6e/users/c4e915efebdf4018bec8e7fc9c010440/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:51.113473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.113766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.114042 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.143238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] MFA Rules not processed for user `c4e915efebdf4018bec8e7fc9c010440`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:51.144459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:51.152596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:51.218791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1301/2622] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:51 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:51.229092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:51.230954 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:51.259243 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:51.316122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb5f1-f5f1-490e-8061-a501e6677d0c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:51.319786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:51.320760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.320760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.321078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.332587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:51.334058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:51.344964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1322/2623] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:51 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:51.389669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:51.391358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:51.478411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6e0a7-8488-4ef7-a970-b01c33307360 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:51.484360 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:51.485208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.485495 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.485778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] PATH_INFO: `/v3/users/f7e198cb3f2a4b0390e25d1678b1ce7b` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.487016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e198cb3f2a4b0390e25d1678b1ce7b)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:51.489196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:51.797448 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e198cb3f2a4b0390e25d1678b1ce7b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:51.798583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e198cb3f2a4b0390e25d1678b1ce7b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:51.810321 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e198cb3f2a4b0390e25d1678b1ce7b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:51.850527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:51.859964 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133caae-2299-437f-85ab-cdc4a328a629 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:51.863668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:51.864275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.864557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.864831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.866596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:51.933186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:51.934667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa06616-8102-4631-9b67-0a7845fa199e service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:51.941826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e198cb3f2a4b0390e25d1678b1ce7b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:51.943828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1323/2624] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:51 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:24:51.955681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e198cb3f2a4b0390e25d1678b1ce7b'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:51.965567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1302/2625] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:51 2022] DELETE /identity/v3/users/f7e198cb3f2a4b0390e25d1678b1ce7b => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:51.970901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:51.971215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:51.971473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:51.993081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:51.994429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:52.071138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1324/2626] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:52.081889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:52.083975 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:52.117715 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:52.150669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de543e1f-39f9-45bd-950f-bc546c21afdc admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:52.154917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:52.155832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:52.156141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:52.156470 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:52.166753 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:52.168256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:52.184011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1303/2627] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:52.680313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:52.681715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:52.693738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecda36-3289-4cb0-98ad-229134aac60a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:52.698316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:52.699281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:52.699573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:52.700617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] PATH_INFO: `/v3/projects/8f504e32d97a4e0aa98f197dd4d35cfa` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:52.701001 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f504e32d97a4e0aa98f197dd4d35cfa)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:52.703759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:52.881044 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:52.888145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f504e32d97a4e0aa98f197dd4d35cfa', 'request_id': 'req-58a9bdab-934b-4b21-937c-f727954abfd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:52.894867 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f504e32d97a4e0aa98f197dd4d35cfa', 'request_id': 'req-58a9bdab-934b-4b21-937c-f727954abfd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.027654 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f504e32d97a4e0aa98f197dd4d35cfa', 'request_id': 'req-58a9bdab-934b-4b21-937c-f727954abfd0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.040767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] The token cache is being invalidate because project 8f504e32d97a4e0aa98f197dd4d35cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:53.041061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.041700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.043229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1325/2628] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:52 2022] DELETE /identity/v3/projects/8f504e32d97a4e0aa98f197dd4d35cfa => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:53.048111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:53.058175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee23c-e894-41f2-a77c-97643b163459 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:53.115404 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:53.116028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:53.116306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:53.116611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] PATH_INFO: `/v3/users/f6069f6622634b048168804a7cc0f5b7` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:53.117785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6069f6622634b048168804a7cc0f5b7)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:53.119757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:53.316281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6069f6622634b048168804a7cc0f5b7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.316904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6069f6622634b048168804a7cc0f5b7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.329626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6069f6622634b048168804a7cc0f5b7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.427764 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6069f6622634b048168804a7cc0f5b7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.435141 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-6e3acbfd-4f06-44f4-a3d3-be13610d9328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6069f6622634b048168804a7cc0f5b7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:53.441292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1304/2629] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:24:53 2022] DELETE /identity/v3/users/f6069f6622634b048168804a7cc0f5b7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:54.053118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:54.054962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:54.070640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9bdab-934b-4b21-937c-f727954abfd0 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:54.129890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:54.130530 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.130841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.131125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] PATH_INFO: `/v3/projects/815e3cf08c8d468688c95b7b28021634` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.132275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=815e3cf08c8d468688c95b7b28021634)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:54.134212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:54.279217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815e3cf08c8d468688c95b7b28021634', 'request_id': 'req-5938d374-1f12-41bb-b4e9-12354aa302ab'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:54.284895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815e3cf08c8d468688c95b7b28021634', 'request_id': 'req-5938d374-1f12-41bb-b4e9-12354aa302ab'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:54.359485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815e3cf08c8d468688c95b7b28021634', 'request_id': 'req-5938d374-1f12-41bb-b4e9-12354aa302ab'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:54.371147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] The token cache is being invalidate because project 815e3cf08c8d468688c95b7b28021634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:54.371414 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:54.372048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-5938d374-1f12-41bb-b4e9-12354aa302ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:54.373517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1326/2630] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:54 2022] DELETE /identity/v3/projects/815e3cf08c8d468688c95b7b28021634 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:54.379185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.379657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.379771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.399884 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:54.401176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:54.463105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1305/2631] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:54.472071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.472365 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.472644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.501061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:54.501061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:54.508921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1327/2632] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:54.510080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:54.518377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:54.522691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ef9a2-35ed-44e1-825f-b751a077336e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:54.527875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:54.528509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.528860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.529133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.529851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:54.530766 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:54.534067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:54.548543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1306/2633] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:24:54.558089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:54.568824 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c118183-43ef-41be-8d1a-3c00aad47ecc None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:54.573927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:54.574573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.574895 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.585444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.585444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:54.585444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:54.704134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1328/2634] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:54 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:54.708621 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:54.719891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd17e3-6c99-43d1-b079-ac21eb6e9d16 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:54.776974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:54.777584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:54.777863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:54.778140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:54.778925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:54.781491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.023540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1307/2635] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:54 2022] POST /identity/v3/users => generated 291 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:55.027768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.039816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7abfa-0838-439c-9eca-d93c3824bebd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.044618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.045229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.045508 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.046337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.047397 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.051687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.064280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1329/2636] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:55.069429 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.080778 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2628b-1960-4bd4-81a3-f5e7cfa61ec8 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.093316 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.093977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.094737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.095038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.095726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.097383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b4d69-99c8-4d3f-b1a4-616d9e2f4eac admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.110390 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1308/2637] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:24:55.118537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.125881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b5923-91ab-4e1a-b637-2e7c3c91ff14 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.130963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.131809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.132130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.132407 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] PATH_INFO: `/v3/projects/534da2a8daab4a8a95751e73531eaecb/users/8ebb40d6951b4a2cbbc057aba43e991c/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.148097 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=534da2a8daab4a8a95751e73531eaecb, user_id=8ebb40d6951b4a2cbbc057aba43e991c, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.150790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.153864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:55.227769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1330/2638] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:55 2022] PUT /identity/v3/projects/534da2a8daab4a8a95751e73531eaecb/users/8ebb40d6951b4a2cbbc057aba43e991c/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:55.238612 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.238996 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.239269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.268366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] MFA Rules not processed for user `8ebb40d6951b4a2cbbc057aba43e991c`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:55.269706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:55.336159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1309/2639] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:55.340700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.403200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e78d2-77a0-4a67-a5d1-04218d5bb4b5 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.408462 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.409071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.409356 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.409635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.410261 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.412208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.538451 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1331/2640] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:55 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:24:55.542541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.554027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4561165-a447-4724-bbc0-88025e161b90 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.611890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.612421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.612706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.612976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.613615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.615554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.754207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1310/2641] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:24:55 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:55.758155 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.769046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed24f9-36b4-41a3-a4b6-b75e2ba51bdf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.773592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.774191 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.774474 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.774781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.781803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.781803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.788178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1332/2642] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:55.792414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.803549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cfffb-b9d5-4d02-860d-7864bdc003be admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.808180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.809127 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.809313 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.809618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] PATH_INFO: `/v3/projects/cf961d6f0c7043c98210f75f84db2860/users/9e0c2526cd38483a8ad274e5a4fcb9a7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.824912 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf961d6f0c7043c98210f75f84db2860, user_id=9e0c2526cd38483a8ad274e5a4fcb9a7, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.828985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.908498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1311/2643] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:55 2022] PUT /identity/v3/projects/cf961d6f0c7043c98210f75f84db2860/users/9e0c2526cd38483a8ad274e5a4fcb9a7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:55.912612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:55.922757 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32bd2f-f8cf-4f48-a4c4-95f80932abcb admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:55.979304 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:55.979915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:55.980197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:55.980471 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:55.981110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:55.982740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7200f7e-6e80-42cf-ba51-3a03b7ae31e5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:55.992831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1333/2644] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:24:55.997299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:56.012353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e99f6-c3a9-4d52-85fe-d03500237bf5 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:56.018283 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:56.018988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:56.019269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:56.019537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] PATH_INFO: `/v3/projects/cf961d6f0c7043c98210f75f84db2860/users/9e0c2526cd38483a8ad274e5a4fcb9a7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:56.021497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf961d6f0c7043c98210f75f84db2860, user_id=9e0c2526cd38483a8ad274e5a4fcb9a7, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:56.024176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:56.035301 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1312/2645] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:24:55 2022] PUT /identity/v3/projects/cf961d6f0c7043c98210f75f84db2860/users/9e0c2526cd38483a8ad274e5a4fcb9a7/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:24:56.046554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:56.046859 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:56.047147 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:56.075466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] MFA Rules not processed for user `9e0c2526cd38483a8ad274e5a4fcb9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:56.076671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:56.145467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1334/2646] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:24:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:56.158121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:56.159855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:56.223228 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb3a1-ec4b-4ed5-bf64-214f960aabda admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:56.227051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:56.227699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:56.227980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:56.228254 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:56.238701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:56.240163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:56.249945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1313/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:56.324811 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:57.742465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:57.876431 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:57.886456 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b95e-a2c7-46ee-9459-c33db2449bd1 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:57.897838 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0c2526cd38483a8ad274e5a4fcb9a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf961d6f0c7043c98210f75f84db2860', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:57.898536 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:57.898825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:57.899109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:57.899781 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:57.900115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:57.901894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:57.920492 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1335/2648] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:24:57.924885 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:57.936458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b6269-7aea-4d4d-93bd-c930c36b3266 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:57.941615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0c2526cd38483a8ad274e5a4fcb9a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf961d6f0c7043c98210f75f84db2860', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:57.942287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:57.942593 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:57.942935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:57.943615 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:57.945568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:58.152543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1314/2649] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:24:57 2022] POST /identity/v3/projects => generated 355 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:24:58.157832 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:58.168093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93dcf-b6fe-4431-9887-30f0f6aa1c1a tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:58.224170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0c2526cd38483a8ad274e5a4fcb9a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf961d6f0c7043c98210f75f84db2860', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:58.224812 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:58.225100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:58.225382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:58.226050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:24:58.226355 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:58.228017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0659d4-300f-4539-b641-4d236a2967c7 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:58.241211 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1336/2650] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:24:58.278950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:58.289378 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5ffd3-a137-448e-9de0-12de39941e1f tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:58.293992 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e0c2526cd38483a8ad274e5a4fcb9a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf961d6f0c7043c98210f75f84db2860', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:58.294681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:58.294961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:58.295287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] PATH_INFO: `/v3/projects/eb715d121eea4d3c82215e1eaca4a230` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:58.296485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eb715d121eea4d3c82215e1eaca4a230)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:58.298373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:58.514710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb715d121eea4d3c82215e1eaca4a230', 'request_id': 'req-79da7b81-a838-4cb8-8653-344cc7d3b163'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:58.681250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797ee2-9631-4a93-9ef1-dce9c5680fb8 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:58.681725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797ee2-9631-4a93-9ef1-dce9c5680fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:58.682094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797ee2-9631-4a93-9ef1-dce9c5680fb8 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:58.683632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1337/2651] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 06:24:58.689382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:58.689626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:58.689907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:58.723372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:58.724560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:58.800465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1338/2652] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:24:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:58.810749 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb715d121eea4d3c82215e1eaca4a230', 'request_id': 'req-79da7b81-a838-4cb8-8653-344cc7d3b163'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:58.813148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:58.814704 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:58.817682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb715d121eea4d3c82215e1eaca4a230', 'request_id': 'req-79da7b81-a838-4cb8-8653-344cc7d3b163'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:58.824849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:24:58.829874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] The token cache is being invalidate because project eb715d121eea4d3c82215e1eaca4a230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:24:58.830161 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:58.830849 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-79da7b81-a838-4cb8-8653-344cc7d3b163 tempest-VolumeQuotaClassesTest-1415622833 tempest-VolumeQuotaClassesTest-1415622833-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:24:58.832367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1315/2653] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:24:58 2022] DELETE /identity/v3/projects/eb715d121eea4d3c82215e1eaca4a230 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:24:58.880074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbbf47-8501-46cc-aa35-c85be783fd21 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:58.883834 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:58.884448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:58.897268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:58.897268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:58.897268 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:58.901443 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:58.902926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e43b03-a1f1-4ba9-b884-a25ad7c566a1 service glance] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:58.914811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1339/2654] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:59.569431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.570011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.570011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] PATH_INFO: `/` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.571359 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1316/2655] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:24:59.576100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049999-7b17-4f76-93c5-3daf11fd0611 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.576391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049999-7b17-4f76-93c5-3daf11fd0611 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.576670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049999-7b17-4f76-93c5-3daf11fd0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.595836 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-25049999-7b17-4f76-93c5-3daf11fd0611 None None] MFA Rules not processed for user `ddd1cf9d4721422194e5ec876b819033`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:59.597042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-25049999-7b17-4f76-93c5-3daf11fd0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:59.647535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1340/2656] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:24:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:24:59.660942 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:59.663304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:59.678466 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cf349-09af-4bca-9026-7cc5e514f955 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:59.683304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:59.684193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.684510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.684737 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.694669 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:59.696181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b4326-2fcf-4534-813d-1dc615f8c7ae service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:59.706304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1317/2657] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:24:59.836416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298a1fb-dccd-464f-aa21-f0687fb74240 None None] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.836948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298a1fb-dccd-464f-aa21-f0687fb74240 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.837047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298a1fb-dccd-464f-aa21-f0687fb74240 None None] PATH_INFO: `/` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.838373 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1341/2658] 10.222.0.40 () {54 vars in 921 bytes} [Fri Aug 26 06:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 06:24:59.844099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.844414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.848683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.866373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] MFA Rules not processed for user `9dd40d8b3cb543feab8424f1d87b7126`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:24:59.867617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:24:59.879400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1318/2659] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Aug 26 06:24:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:24:59.890610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298a1fb-dccd-464f-aa21-f0687fb74240 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:24:59.901997 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298a1fb-dccd-464f-aa21-f0687fb74240 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:24:59.906294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:24:59.906930 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:24:59.907220 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:24:59.907500 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:24:59.918279 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:24:59.919767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cf414-c225-4e66-9fb9-86c4914a3f03 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:24:59.931687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1342/2660] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:24:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:00.113172 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:00.230255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:00.231698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:00.299724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb3ad0-e9b4-4b4c-88a1-5c7bef909d4b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:00.304694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:00.305327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:00.305603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:00.305871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] PATH_INFO: `/v3/users/8ebb40d6951b4a2cbbc057aba43e991c` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:00.307005 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebb40d6951b4a2cbbc057aba43e991c)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:00.308905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:00.584411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebb40d6951b4a2cbbc057aba43e991c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:00.585306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebb40d6951b4a2cbbc057aba43e991c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:00.599674 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebb40d6951b4a2cbbc057aba43e991c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:00.868292 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebb40d6951b4a2cbbc057aba43e991c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:00.878568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebb40d6951b4a2cbbc057aba43e991c'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:00.884769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1319/2661] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:00 2022] DELETE /identity/v3/users/8ebb40d6951b4a2cbbc057aba43e991c => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:00.889118 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:00.889608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:00.890074 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:00.909910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:00.911168 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:00.977115 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1343/2662] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:00.988543 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:00.990060 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:01.074239 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ffcda-a3b1-4b80-be34-4c63d03c343d admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:01.078248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:01.078900 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:01.079185 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:01.079462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:01.088858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:01.090328 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:01.100427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1320/2663] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:01.658589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:01.660291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:01.674084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbaaad-5631-4f86-9840-8927ef8b03f4 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:01.679302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:01.679972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:01.680259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:01.680545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] PATH_INFO: `/v3/projects/534da2a8daab4a8a95751e73531eaecb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:01.681722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=534da2a8daab4a8a95751e73531eaecb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:01.683780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:01.913413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534da2a8daab4a8a95751e73531eaecb', 'request_id': 'req-6e6f9088-8fd4-44d9-87c1-baee7c088c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:01.918947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534da2a8daab4a8a95751e73531eaecb', 'request_id': 'req-6e6f9088-8fd4-44d9-87c1-baee7c088c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.448237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534da2a8daab4a8a95751e73531eaecb', 'request_id': 'req-6e6f9088-8fd4-44d9-87c1-baee7c088c55'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.448237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] The token cache is being invalidate because project 534da2a8daab4a8a95751e73531eaecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:02.448237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.448237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.448237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1344/2664] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:01 2022] DELETE /identity/v3/projects/534da2a8daab4a8a95751e73531eaecb => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:02.451191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:02.464164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939d4f-11f3-4c4b-8562-62e6480c9b61 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:02.543491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:02.544281 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:02.544668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:02.545011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] PATH_INFO: `/v3/users/9e0c2526cd38483a8ad274e5a4fcb9a7` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:02.546802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0c2526cd38483a8ad274e5a4fcb9a7)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:02.548769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:02.706180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:02.886376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0c2526cd38483a8ad274e5a4fcb9a7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.887274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0c2526cd38483a8ad274e5a4fcb9a7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:02.904748 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0c2526cd38483a8ad274e5a4fcb9a7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:03.051364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0c2526cd38483a8ad274e5a4fcb9a7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:03.059198 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0c2526cd38483a8ad274e5a4fcb9a7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:03.066602 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1321/2665] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:02 2022] DELETE /identity/v3/users/9e0c2526cd38483a8ad274e5a4fcb9a7 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:03.800558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:03.800558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:03.820481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f9088-8fd4-44d9-87c1-baee7c088c55 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:03.829934 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:03.844140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:03.845765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:03.914467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:03.915315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:03.915717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:03.916104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] PATH_INFO: `/v3/projects/cf961d6f0c7043c98210f75f84db2860` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:03.917366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf961d6f0c7043c98210f75f84db2860)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:03.919606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:03.923395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adb7e-edb2-4f09-89be-74b4a3e6ef48 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:03.928545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:03.929205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:03.929485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:03.929756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] PATH_INFO: `/v3/users/b7ce0d49e0e14d1fbc103ae528c05e60` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:03.930904 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ce0d49e0e14d1fbc103ae528c05e60)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:03.932816 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:04.080971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf961d6f0c7043c98210f75f84db2860', 'request_id': 'req-16e1cb7a-8734-4dcc-b189-4e218960474c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.086589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf961d6f0c7043c98210f75f84db2860', 'request_id': 'req-16e1cb7a-8734-4dcc-b189-4e218960474c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.227362 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf961d6f0c7043c98210f75f84db2860', 'request_id': 'req-16e1cb7a-8734-4dcc-b189-4e218960474c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.240084 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] The token cache is being invalidate because project cf961d6f0c7043c98210f75f84db2860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:04.240375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.241016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-16e1cb7a-8734-4dcc-b189-4e218960474c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.242498 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1345/2666] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:03 2022] DELETE /identity/v3/projects/cf961d6f0c7043c98210f75f84db2860 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:04.246727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca22f4-4d3c-4d65-8b86-2f8531916d9b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.246727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca22f4-4d3c-4d65-8b86-2f8531916d9b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.246727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca22f4-4d3c-4d65-8b86-2f8531916d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.264294 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-5eca22f4-4d3c-4d65-8b86-2f8531916d9b None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:04.265816 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5eca22f4-4d3c-4d65-8b86-2f8531916d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.321442 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ce0d49e0e14d1fbc103ae528c05e60'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.321671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1346/2667] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Aug 26 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:04.322261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ce0d49e0e14d1fbc103ae528c05e60'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.328013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.328308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.328592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.341069 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ce0d49e0e14d1fbc103ae528c05e60'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.352487 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:04.353699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.417847 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1347/2668] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:04.418852 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:04.420393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.473395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ce0d49e0e14d1fbc103ae528c05e60'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.481503 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-0470c24a-f2bb-4901-af5e-90c1ca8eb9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ce0d49e0e14d1fbc103ae528c05e60'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:04.492406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1322/2669] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:03 2022] DELETE /identity/v3/users/b7ce0d49e0e14d1fbc103ae528c05e60 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:04.492406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56061-735a-47ca-95c1-301bc17c7b0a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.492406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56061-735a-47ca-95c1-301bc17c7b0a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.492406 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e56061-735a-47ca-95c1-301bc17c7b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.494771 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf50e2-088e-4139-86f0-b15e814d8586 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:04.512215 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d9e56061-735a-47ca-95c1-301bc17c7b0a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:04.513695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d9e56061-735a-47ca-95c1-301bc17c7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.564758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:04.565387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.565829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.566258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.568282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.594950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1323/2670] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:04.594950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.594950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.594950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.620072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:04.621389 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.638247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1324/2671] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:04.638247 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:04.642779 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ce5d0b-b684-4789-98cc-56bad2387aff None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:04.644592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:04.646389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:04.649531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:04.650155 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.650587 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.650901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.651562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:25:04.651870 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:04.653686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:04.658952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1348/2672] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 06:25:04.659826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:04.661494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:04.668075 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1325/2673] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:25:04.677263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:04.698611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa635e0a-d369-40e7-a43f-9bfc4df1f03a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:04.712236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:04.712236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.712236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.712236 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.712630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:04.712630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:04.730033 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2cad-03b6-41ce-b138-469757b54f07 service swift] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:04.734347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:04.734996 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:04.735288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:04.735566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:04.744753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:04.746236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:04.756668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1349/2674] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:05.177180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:05.178196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1326/2675] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:04 2022] POST /identity/v3/projects => generated 385 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:05.198887 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4638b67-bae5-47db-95d2-d0a481112213 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:05.331535 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:05.332215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.332552 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.332881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:05.333576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:05.336165 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:05.580551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1350/2676] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:05 2022] POST /identity/v3/users => generated 293 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:05.585485 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:05.599537 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8fa9c-3fa1-4fc3-9576-b5c226eab3a1 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:05.605787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:05.606858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.607373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.607871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:05.609630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:05.611902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:05.632502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1327/2677] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:05.636240 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:05.642491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:05.645093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:05.648231 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24389ab-87bf-4ba1-a148-3a84554999d7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:05.653313 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:05.653931 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.654222 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.654504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:05.658673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:05.658673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:05.675287 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1351/2678] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:05.677240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941d4da-b69c-48b6-9037-1d4c9892e385 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:05.679427 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:05.689619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:05.691115 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.695583 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.696740 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] PATH_INFO: `/v3/projects/9a46d2e7309e4b6883b07521a29d8b33` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:05.698857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a46d2e7309e4b6883b07521a29d8b33)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:05.702253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:05.719139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83517188-4d32-455d-9604-90b31ce19e94 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:05.726235 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:05.736395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.736722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.737008 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] PATH_INFO: `/v3/projects/854f731ec60b4298a999377934c541e2/users/d091e6c835ac4d8e9b634cf1d693be57/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:05.779468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] RBAC: Authorizing `identity:create_grant(project_id=854f731ec60b4298a999377934c541e2, user_id=d091e6c835ac4d8e9b634cf1d693be57, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:05.783216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74bff4-043a-4cd8-bdf3-e566edbf786e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:05.975901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:05.982282 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1352/2679] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:05 2022] PUT /identity/v3/projects/854f731ec60b4298a999377934c541e2/users/d091e6c835ac4d8e9b634cf1d693be57/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:05.992251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a46d2e7309e4b6883b07521a29d8b33', 'request_id': 'req-2075990d-41bc-4061-b0d4-58bd5a546963'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:05.995284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:05.995619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:05.995935 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:06.036448 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] MFA Rules not processed for user `d091e6c835ac4d8e9b634cf1d693be57`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:06.050569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.116510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1353/2680] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:06.129825 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:06.131352 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.144860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a46d2e7309e4b6883b07521a29d8b33', 'request_id': 'req-2075990d-41bc-4061-b0d4-58bd5a546963'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.147625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a46d2e7309e4b6883b07521a29d8b33', 'request_id': 'req-2075990d-41bc-4061-b0d4-58bd5a546963'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.161600 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] The token cache is being invalidate because project 9a46d2e7309e4b6883b07521a29d8b33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:06.161945 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.162694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.164395 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1328/2681] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:05 2022] DELETE /identity/v3/projects/9a46d2e7309e4b6883b07521a29d8b33 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:06.168170 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:06.169632 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.211227 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b484-d652-473a-b2a1-2ceac7e6d77e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:06.217501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:06.218200 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:06.218591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:06.218914 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:06.221332 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.232835 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:06.235310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:06.244913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1354/2682] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:25:06.271328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:06.273069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.287103 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075990d-41bc-4061-b0d4-58bd5a546963 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:06.290010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7d00-8ed5-4b38-bb64-32a03fe77c10 service cinder] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:06.292592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:06.293337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:06.293684 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:06.294016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] PATH_INFO: `/v3/users/c4e915efebdf4018bec8e7fc9c010440` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:06.296891 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e915efebdf4018bec8e7fc9c010440)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:06.297436 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:06.298433 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:06.299095 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:06.299575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] PATH_INFO: `/v3/users/d091e6c835ac4d8e9b634cf1d693be57` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:06.300685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:06.301861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d091e6c835ac4d8e9b634cf1d693be57)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:06.305545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:06.602122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e915efebdf4018bec8e7fc9c010440'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.603227 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e915efebdf4018bec8e7fc9c010440'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.608628 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd091e6c835ac4d8e9b634cf1d693be57'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.609680 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd091e6c835ac4d8e9b634cf1d693be57'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.620044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e915efebdf4018bec8e7fc9c010440'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.624537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd091e6c835ac4d8e9b634cf1d693be57'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.752394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e915efebdf4018bec8e7fc9c010440'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.760118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41ed07e2-8794-4b25-8785-f0069c0cc841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e915efebdf4018bec8e7fc9c010440'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.766585 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1329/2683] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:06 2022] DELETE /identity/v3/users/c4e915efebdf4018bec8e7fc9c010440 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:06.822264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd091e6c835ac4d8e9b634cf1d693be57'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.833302 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd091e6c835ac4d8e9b634cf1d693be57'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:06.849267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1355/2684] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:06 2022] DELETE /identity/v3/users/d091e6c835ac4d8e9b634cf1d693be57 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:06.854699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:06.854927 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:06.855193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:06.875530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:06.877031 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:06.950483 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1330/2685] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:06.962733 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:06.964674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:07.040871 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d29dd-9421-4c34-aaf2-fefdd6eee3c2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:07.045212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:07.046054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:07.046718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:07.047104 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:07.060609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:07.062773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:07.074982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1356/2686] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:07.119394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:07.494409 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:07.496726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:07.515736 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d9d0-f95c-4b20-a9c7-0fdd889dce3f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:07.521539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:07.522549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:07.523113 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:07.523788 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] PATH_INFO: `/v3/projects/7682e0952d07486086a06ab01572bc6e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:07.525411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7682e0952d07486086a06ab01572bc6e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:07.528354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:07.742735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7682e0952d07486086a06ab01572bc6e', 'request_id': 'req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:07.864905 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:07.869785 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:07.894555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec7041-c63b-40a9-a9ac-958712391110 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:07.930874 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7682e0952d07486086a06ab01572bc6e', 'request_id': 'req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:07.939881 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7682e0952d07486086a06ab01572bc6e', 'request_id': 'req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:07.964090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] The token cache is being invalidate because project 7682e0952d07486086a06ab01572bc6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:07.964812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:07.965677 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-86cd0d69-9b82-4a83-8b77-801ae6ff68a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:07.969050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1331/2687] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:07 2022] DELETE /identity/v3/projects/7682e0952d07486086a06ab01572bc6e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:07.974928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4aa5-c717-4ee4-8d28-45fbdf5b1c40 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:07.975264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4aa5-c717-4ee4-8d28-45fbdf5b1c40 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:07.975493 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c4aa5-c717-4ee4-8d28-45fbdf5b1c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.000295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.000880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.001167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.001446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] PATH_INFO: `/v3/projects/854f731ec60b4298a999377934c541e2` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.002727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854f731ec60b4298a999377934c541e2)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.004700 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.006370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5e5c4aa5-c717-4ee4-8d28-45fbdf5b1c40 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:08.007948 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5e5c4aa5-c717-4ee4-8d28-45fbdf5b1c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.018905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1332/2688] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:25:08.031671 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.031961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.032241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.055552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:08.057224 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.068771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1333/2689] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:25:08.073197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.083966 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa757-4e78-4fcf-bcf1-f8b8c9085a1b None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.089030 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.093356 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.105723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1334/2690] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:25:08.110222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.120572 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46d47-37c6-402d-8c1e-236e5f2d1f1c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.126527 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.127265 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.127555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.127867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.128596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.130407 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.147381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854f731ec60b4298a999377934c541e2', 'request_id': 'req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:08.154387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854f731ec60b4298a999377934c541e2', 'request_id': 'req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:08.165196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:08.253033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1335/2691] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:08.257221 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.273820 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854f731ec60b4298a999377934c541e2', 'request_id': 'req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:08.310782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] The token cache is being invalidate because project 854f731ec60b4298a999377934c541e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:08.311179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:08.315071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-f93c94b7-bf3c-43cb-a8cd-da049ff579c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:08.316734 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1357/2692] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:07 2022] DELETE /identity/v3/projects/854f731ec60b4298a999377934c541e2 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:08.322210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204d96-1f30-4668-92f9-7c59bf433649 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.322696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204d96-1f30-4668-92f9-7c59bf433649 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.323674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204d96-1f30-4668-92f9-7c59bf433649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.355977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9a204d96-1f30-4668-92f9-7c59bf433649 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:08.357259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9a204d96-1f30-4668-92f9-7c59bf433649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.391440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de15c7-3d24-412e-ad95-ec992692d693 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.399559 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.400726 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.401179 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.408091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.408091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.408091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.494361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1358/2693] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:25:08.501971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.502269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.502749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.524519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:08.525675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.536882 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1336/2694] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:08.542804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1359/2695] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:08.543011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.544052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.547094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.559224 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59584047-b0d4-466e-8eab-32e232a1473f None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.565595 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b9e3f-c1b6-46cd-a9e5-0514adbf6247 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.570239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.571217 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.571619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.571910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.572764 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:25:08.578646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.584710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.584710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.584710 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.585045 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.585134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.585134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.585134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.585360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=100 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:08.597918 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1337/2696] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:08.601218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.606672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1360/2697] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:25:08.610152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.617522 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb02a8-dbc6-408b-8eda-3d8a99dcf169 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.620961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9912b-c328-437c-85e8-3dc581555d72 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.623138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.623270 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.625171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.625462 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.626541 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.627263 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.627435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.627692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.628042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.628770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.630805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.630984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.654663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1338/2698] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:08.657582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.668314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b59a9-434b-4aeb-afe6-098b921b189c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.673691 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.674447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.674844 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.675182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] PATH_INFO: `/v3/projects/08024d4f339d4a0ba038ead8dce6d522/users/73bb59b4a70f4ef9acfd4ee3ae72cab3/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.693790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] RBAC: Authorizing `identity:create_grant(project_id=08024d4f339d4a0ba038ead8dce6d522, user_id=73bb59b4a70f4ef9acfd4ee3ae72cab3, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:08.696762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750073-7384-4438-bc8f-f03185f5969e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:08.801641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1361/2699] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:08.805650 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:08.839486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:08.850733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1339/2700] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:08 2022] PUT /identity/v3/projects/08024d4f339d4a0ba038ead8dce6d522/users/73bb59b4a70f4ef9acfd4ee3ae72cab3/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:08.858262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.858545 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.859146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:08.893108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] MFA Rules not processed for user `73bb59b4a70f4ef9acfd4ee3ae72cab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:08.894472 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:08.903767 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce207cfb-48e1-4eae-bbbb-e2ef6503b603 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:08.997459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1340/2701] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:25:08.998283 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:08.998988 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:08.999303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:08.999726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.000364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.001944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.002560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.018282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d57cfe-28b4-4b4e-866e-44b3f7f06bb3 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.023555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.024293 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.024642 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.024985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.025688 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.028173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.032959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:09.176214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1362/2702] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:08 2022] POST /identity/v3/users => generated 293 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:09.177430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.189138 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba1ebf-6fc1-4ab3-9f91-169466e590c8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.194392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.195076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.195437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.195750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.196559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.198186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.209489 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1363/2703] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:09.223691 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.225344 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1341/2704] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:09 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:09.238077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef567-a52b-475f-b0c2-eb8286600f3f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.241131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.310791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.312347 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.312725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.314028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.314830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.318354 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.327959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1364/2705] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:09.333241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92a966-24b3-48bb-b140-8eaf34b200a6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.335635 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.346403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.346971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beec38e-2226-43fc-bc25-32f146850f61 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.360118 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.360295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.360433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.360433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.360616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.360616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.360794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.360892 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] PATH_INFO: `/v3/projects/be95e32378f94fb3ba17c682feb572aa/users/cd532f163c4345aeb0bd0e9e14ec431d/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.360983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.375184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=be95e32378f94fb3ba17c682feb572aa, user_id=cd532f163c4345aeb0bd0e9e14ec431d, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.378531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd32d10-80f7-46f0-803c-b38dfa20c3ca admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.526468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:09.534719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1342/2706] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:09 2022] POST /identity/v3/users => generated 294 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:09.535689 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.537384 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1365/2707] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:09 2022] PUT /identity/v3/projects/be95e32378f94fb3ba17c682feb572aa/users/cd532f163c4345aeb0bd0e9e14ec431d/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:09.552519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.552768 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.553052 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.558444 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6f84-5c4a-417a-8cef-167b0b14fefc admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.605199 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] MFA Rules not processed for user `cd532f163c4345aeb0bd0e9e14ec431d`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:09.607309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:09.652591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.653574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.654017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.654459 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.656469 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.658358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.669705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1343/2708] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:09.674643 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.694255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f1fcd-ce2a-4d34-bdbd-755bb2a5dd09 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.700563 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.701240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.701523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.701896 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] PATH_INFO: `/v3/projects/49e1c85451be406da11e6db1c4e85c44/users/b775e3cd0172467b8b65f348aec2a899/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.728291 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e1c85451be406da11e6db1c4e85c44, user_id=b775e3cd0172467b8b65f348aec2a899, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.738841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.743788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1366/2709] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:09.748878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.766005 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610306c-5dc6-4f6e-9376-666b8c17cbb3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.771339 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.771963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.772251 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.772578 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.773326 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:09.775303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:09.852097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1344/2710] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:09 2022] PUT /identity/v3/projects/49e1c85451be406da11e6db1c4e85c44/users/b775e3cd0172467b8b65f348aec2a899/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:09.856350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.907527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1367/2711] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:09 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:09.911769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:09.964639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:09.969698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeccdc2-046e-41dc-a6e3-0ea283aa9d09 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.986952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1223c15-94a4-44fa-96c3-e3ffc7569239 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:09.996391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:09.996391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:09.996391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:09.996391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:09.996933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.000537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.036999 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.037952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.038473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.038913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.039762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.042124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.054782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1345/2712] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:10.059039 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.069569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f29155-97bc-4507-85c0-0c7d1679d917 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.074394 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.075082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.075366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.075690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] PATH_INFO: `/v3/projects/49e1c85451be406da11e6db1c4e85c44/users/b775e3cd0172467b8b65f348aec2a899/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.078352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e1c85451be406da11e6db1c4e85c44, user_id=b775e3cd0172467b8b65f348aec2a899, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.081045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f0b0d-e2c3-4dde-baf8-fd4e66469984 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.094180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1346/2713] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:10 2022] PUT /identity/v3/projects/49e1c85451be406da11e6db1c4e85c44/users/b775e3cd0172467b8b65f348aec2a899/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:25:10.104422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.104711 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.104990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.125388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1368/2714] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:09 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:10.129375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.140481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb1928-9d07-4b81-bcb6-81d8b6760857 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.141964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] MFA Rules not processed for user `b775e3cd0172467b8b65f348aec2a899`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:10.143624 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:10.147653 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:10.249984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1347/2715] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:25:10.256605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.257490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.258002 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.261116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.261662 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.262746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.262988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:10.265583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.280830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1369/2716] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:25:10.287888 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.303561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af0855-cedb-4d36-8fbc-163c00d18482 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.315587 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.316348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.316762 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.316903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.317555 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.319219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.332029 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1370/2717] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:10.336490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:10.337797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.356593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dc23b-f6b5-41c0-9611-e8f55d832b4e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.360431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d433726-b9b3-470f-8983-d3e741bade69 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.367504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.368193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.368479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.368766 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] PATH_INFO: `/v3/projects/5731cc4470cf401293268f89e053f514/users/9024a3dcc45049da9dd13aa2bac23fa6/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.370182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.370832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.371164 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.371374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.399699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5731cc4470cf401293268f89e053f514, user_id=9024a3dcc45049da9dd13aa2bac23fa6, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.403512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.459051 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.460682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0840f6-abcf-42e2-a473-b93b2665dfa7 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.470979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1348/2718] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:10.522718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1371/2719] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:10 2022] PUT /identity/v3/projects/5731cc4470cf401293268f89e053f514/users/9024a3dcc45049da9dd13aa2bac23fa6/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:10.528867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.528867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.528867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.558062 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:10.560027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] MFA Rules not processed for user `9024a3dcc45049da9dd13aa2bac23fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:10.563009 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:10.638489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:10.689183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1349/2720] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:10.694378 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:10.773198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f9ac4-319f-48ed-89b0-6c51b02f5dba admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:10.780978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:10.781618 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:10.781908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:10.782400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:10.783077 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:10.785175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:10.990221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1372/2721] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:10 2022] POST /identity/v3/projects => generated 385 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:10.994480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.006596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f13cf3-a56e-46ad-838d-feb9c38d3a2f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.070917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.071701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.072086 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.072460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.073200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.075261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.194539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1350/2722] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:10 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:11.198442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.210035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea630-a4f9-492d-a62f-f50500aaecbf admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.215209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.215860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.216121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.216403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.217105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.218790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.230869 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1373/2723] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:11.235207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.245854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8323e4-dc56-4ff7-b1d3-525afcd74714 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.251576 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.252608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.252968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.253318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] PATH_INFO: `/v3/projects/4afed1a162ea49468f0e9058ad5d479b/users/8a5bf8956207467f95d56333269191a6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.277685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4afed1a162ea49468f0e9058ad5d479b, user_id=8a5bf8956207467f95d56333269191a6, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.277685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.397637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1351/2724] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:11 2022] PUT /identity/v3/projects/4afed1a162ea49468f0e9058ad5d479b/users/8a5bf8956207467f95d56333269191a6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:11.402537 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.413101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60649f6d-a4f1-4979-b2d2-ecb8f74e8265 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.492559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.493466 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.494142 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.494486 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.495248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.497129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c850bd-2ede-4522-8c4a-b4f5bbbed490 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.509503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1374/2725] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:25:11.516871 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.530769 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562049dd-5df7-42a1-ac01-6b0c5521d6e6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.538250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.539166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.539453 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.539727 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] PATH_INFO: `/v3/projects/4afed1a162ea49468f0e9058ad5d479b/users/8a5bf8956207467f95d56333269191a6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.541781 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4afed1a162ea49468f0e9058ad5d479b, user_id=8a5bf8956207467f95d56333269191a6, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.544746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.557300 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1352/2726] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:11 2022] PUT /identity/v3/projects/4afed1a162ea49468f0e9058ad5d479b/users/8a5bf8956207467f95d56333269191a6/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:11.568120 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.568437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.568746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.609015 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] MFA Rules not processed for user `8a5bf8956207467f95d56333269191a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:11.610688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:11.709778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1375/2727] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:11.730794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:11.806958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8273b-a735-46cc-a5d4-6400bf2a4d4a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:11.810723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:11.811952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:11.812260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:11.812542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:11.904186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:11.906025 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:11.920437 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1353/2728] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:11.991300 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:12.553553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:15.384925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:15.452337 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7edbde-d510-4bc2-b313-c085ae3d1d16 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:15.458110 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:15.458789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:15.459248 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:15.459546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] PATH_INFO: `/v3/users/73bb59b4a70f4ef9acfd4ee3ae72cab3` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:15.460770 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=73bb59b4a70f4ef9acfd4ee3ae72cab3)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:15.463250 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:15.469789 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:15.797290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:15.799480 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:15.805191 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10caac2-075c-4b1b-be49-cdbcd7584f71 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:15.809084 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:15.809713 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:15.809988 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:15.810256 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:15.868841 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:15.870334 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6f88-a6e7-4c67-b244-21df951594cb service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:15.880129 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1354/2729] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:25:15.897468 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73bb59b4a70f4ef9acfd4ee3ae72cab3'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:15.898205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73bb59b4a70f4ef9acfd4ee3ae72cab3'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:15.922214 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73bb59b4a70f4ef9acfd4ee3ae72cab3'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:16.176293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73bb59b4a70f4ef9acfd4ee3ae72cab3'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:16.183902 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73bb59b4a70f4ef9acfd4ee3ae72cab3'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:16.190490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1376/2730] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:15 2022] DELETE /identity/v3/users/73bb59b4a70f4ef9acfd4ee3ae72cab3 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:16.195152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:16.195441 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:16.195716 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:16.216263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:16.218327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:16.277464 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1355/2731] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:16.287419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:16.288879 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:16.347004 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5879ae-b576-44c8-a958-6fee14d4c73b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:16.350795 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:16.351412 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:16.351699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:16.351982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:16.361593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:16.363091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:16.372759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1377/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:16.971964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:16.975166 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:16.992147 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f74c2-407e-4902-bbc2-7dca30c3b138 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:16.997114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:16.997758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:16.998034 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:16.998905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] PATH_INFO: `/v3/projects/08024d4f339d4a0ba038ead8dce6d522` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:17.002422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08024d4f339d4a0ba038ead8dce6d522)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:17.004509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:17.374979 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08024d4f339d4a0ba038ead8dce6d522', 'request_id': 'req-5925cc3f-035f-40d1-b9fa-649d060b5236'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:17.622036 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08024d4f339d4a0ba038ead8dce6d522', 'request_id': 'req-5925cc3f-035f-40d1-b9fa-649d060b5236'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:17.629434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08024d4f339d4a0ba038ead8dce6d522', 'request_id': 'req-5925cc3f-035f-40d1-b9fa-649d060b5236'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:17.641893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] The token cache is being invalidate because project 08024d4f339d4a0ba038ead8dce6d522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:17.642186 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:17.642897 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:17.644405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1356/2733] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:16 2022] DELETE /identity/v3/projects/08024d4f339d4a0ba038ead8dce6d522 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:17.648506 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:17.658464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e878d7-aa68-481c-94ee-ecfb916aaabe service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:17.730121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:17.730962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:17.731449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:17.731828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] PATH_INFO: `/v3/users/b775e3cd0172467b8b65f348aec2a899` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:17.733701 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b775e3cd0172467b8b65f348aec2a899)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:17.736183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:18.104229 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b775e3cd0172467b8b65f348aec2a899'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:18.104913 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b775e3cd0172467b8b65f348aec2a899'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:18.118788 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b775e3cd0172467b8b65f348aec2a899'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:18.295065 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b775e3cd0172467b8b65f348aec2a899'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:18.298336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:18.306405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-22933978-f601-44c6-9c1f-b402118ef68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b775e3cd0172467b8b65f348aec2a899'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:18.314226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1378/2734] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:17 2022] DELETE /identity/v3/users/b775e3cd0172467b8b65f348aec2a899 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:18.620954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:18.620954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:18.641694 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925cc3f-035f-40d1-b9fa-649d060b5236 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:18.692641 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:18.693262 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:18.718107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:18.718107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:18.718107 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:18.760784 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:18.762220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:18.772983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1357/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:18.821782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:18.821782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:18.821782 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:18.840366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] MFA Rules not processed for user `4ecc0db48a09441eb68340c620de4263`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:18.841599 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:18.901458 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1379/2736] 10.222.0.40 () {58 vars in 967 bytes} [Fri Aug 26 06:25:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:18.918065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:18.919530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:18.996197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968f09-bb27-4d6e-bfa4-27613faa4cef service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:19.001364 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d3c0f71e8984988b432390ec99977de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:19.001997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.002280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.002558 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.011038 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:19.012622 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea885f-a401-4e40-8c21-1da53deda85b service swift] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:19.021562 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1358/2737] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Aug 26 06:25:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 06:25:19.043935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:19.055181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:19.055181 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:19.143435 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2f3b-ffd2-4d76-84eb-12b4c61adab4 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:19.148447 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:19.149088 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.149372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.150213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] PATH_INFO: `/v3/projects/49e1c85451be406da11e6db1c4e85c44` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.151740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e1c85451be406da11e6db1c4e85c44)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:19.153657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:19.359124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e1c85451be406da11e6db1c4e85c44', 'request_id': 'req-58f77fb4-78c8-43f2-b8da-654189df83fa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:19.366819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e1c85451be406da11e6db1c4e85c44', 'request_id': 'req-58f77fb4-78c8-43f2-b8da-654189df83fa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:19.576061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e1c85451be406da11e6db1c4e85c44', 'request_id': 'req-58f77fb4-78c8-43f2-b8da-654189df83fa'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:19.589374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] The token cache is being invalidate because project 49e1c85451be406da11e6db1c4e85c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:19.589671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:19.590341 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-58f77fb4-78c8-43f2-b8da-654189df83fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:19.591899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1380/2738] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:19 2022] DELETE /identity/v3/projects/49e1c85451be406da11e6db1c4e85c44 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:19.600805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.601263 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.602020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.626813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:19.627800 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:19.698362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1359/2739] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:19.707604 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.707894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.708176 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.728505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:19.729681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:19.738517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1381/2740] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:19.742819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:19.754369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac08b1-8d69-4e64-b703-6d9be4b5fc01 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:19.759350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:19.759960 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.760240 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.760510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.761152 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:25:19.761452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:19.763260 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:19.776396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1360/2741] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:25:19.781802 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:19.791531 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd096b-e2d0-49c7-abfe-69e31a8f0c9a None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:19.797162 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:19.797815 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:19.802418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:19.802418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:19.802418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:19.802418 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:19.938860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1382/2742] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:19 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:19.943094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:19.953494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d36a32-e390-48e4-85c2-29bea19f78f6 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:20.013353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:20.014044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.014408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.014734 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.015422 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:20.017350 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:20.271795 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1361/2743] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:19 2022] POST /identity/v3/users => generated 284 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:20.273061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:20.284962 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef00644-c1a3-48d7-b173-cef0281a4745 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:20.290382 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:20.291219 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.291532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.291813 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.292605 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:20.294546 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:20.306130 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1383/2744] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:20.310733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:20.325752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5a594-6808-478f-9bb4-fa27e1a847b7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:20.334729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:20.335352 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.335657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.336131 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.336791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:20.338530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ca871-c525-4e45-83b6-5b4f9f63c99e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:20.351314 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1362/2745] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:25:20.355708 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:20.367631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7ff23-2c2a-4848-b87e-32cc5f62443a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:20.373125 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:20.374983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.375614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.375614 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] PATH_INFO: `/v3/projects/18b43e784f8d4c85bb1d714b6f678b37/users/da3b758503fd46788a6dd0f1b1d153b8/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.395818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b43e784f8d4c85bb1d714b6f678b37, user_id=da3b758503fd46788a6dd0f1b1d153b8, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:20.399481 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:20.576595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1384/2746] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:20 2022] PUT /identity/v3/projects/18b43e784f8d4c85bb1d714b6f678b37/users/da3b758503fd46788a6dd0f1b1d153b8/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:20.587064 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.587358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.587640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.616554 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] MFA Rules not processed for user `da3b758503fd46788a6dd0f1b1d153b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:20.617845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:20.707658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1363/2747] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:20.720870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:20.723921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:20.798673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486778fe-fcab-43ac-bf88-36f34156d4a8 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:20.798673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee70da9d2f4451ba0d2133653b7d144', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:20.798673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:20.798673 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:20.799076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:20.806460 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:20.808080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db73c362-7e32-4de1-a8cd-8ea1f0411509 service glance] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:20.817256 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1385/2748] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:21.797575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:22.235556 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=100 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:23.690830 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:24.240963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:24.242391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:24.262436 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:24.310530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa0a8c-4407-4208-875a-32d913d3ad18 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:24.315977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:24.316633 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:24.316920 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:24.317201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] PATH_INFO: `/v3/users/cd532f163c4345aeb0bd0e9e14ec431d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:24.318369 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd532f163c4345aeb0bd0e9e14ec431d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:24.320370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:24.645330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd532f163c4345aeb0bd0e9e14ec431d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:24.646108 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd532f163c4345aeb0bd0e9e14ec431d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:24.662214 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd532f163c4345aeb0bd0e9e14ec431d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:24.845176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd532f163c4345aeb0bd0e9e14ec431d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:24.853420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd532f163c4345aeb0bd0e9e14ec431d'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:24.860243 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1364/2749] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:24 2022] DELETE /identity/v3/users/cd532f163c4345aeb0bd0e9e14ec431d => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:24.864854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:24.865380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:24.865567 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:24.884860 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:24.886100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:24.955716 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1386/2750] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:24.965501 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:24.967027 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:25.030299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0097-80e4-41e5-906c-0038d2f4f9ed admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:25.034509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:25.035175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:25.037353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:25.037353 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:25.045474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:25.046961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:25.058420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1365/2751] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:25.782564 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:25.784317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:25.800221 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e0f34-96fb-4231-a05d-41589963bad5 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:25.805516 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:25.806841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:25.807316 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:25.807603 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] PATH_INFO: `/v3/projects/be95e32378f94fb3ba17c682feb572aa` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:25.809170 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=be95e32378f94fb3ba17c682feb572aa)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:25.815450 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:25.977642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be95e32378f94fb3ba17c682feb572aa', 'request_id': 'req-74236910-3b1c-4dcc-8134-926c49bc30ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:25.983438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be95e32378f94fb3ba17c682feb572aa', 'request_id': 'req-74236910-3b1c-4dcc-8134-926c49bc30ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.073522 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be95e32378f94fb3ba17c682feb572aa', 'request_id': 'req-74236910-3b1c-4dcc-8134-926c49bc30ff'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.092746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] The token cache is being invalidate because project be95e32378f94fb3ba17c682feb572aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:26.093043 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.093791 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.095395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1387/2752] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:25 2022] DELETE /identity/v3/projects/be95e32378f94fb3ba17c682feb572aa => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:26.099617 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:26.110439 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f902f-cd2b-475e-93db-2c4fa6e222b8 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:26.149349 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:26.172590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:26.173397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:26.173757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:26.173929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] PATH_INFO: `/v3/users/9024a3dcc45049da9dd13aa2bac23fa6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:26.175187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9024a3dcc45049da9dd13aa2bac23fa6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:26.177362 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:26.488323 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9024a3dcc45049da9dd13aa2bac23fa6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.488990 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9024a3dcc45049da9dd13aa2bac23fa6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.501890 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9024a3dcc45049da9dd13aa2bac23fa6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.576011 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9024a3dcc45049da9dd13aa2bac23fa6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.584377 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9024a3dcc45049da9dd13aa2bac23fa6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:26.590309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1366/2753] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:26 2022] DELETE /identity/v3/users/9024a3dcc45049da9dd13aa2bac23fa6 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:27.152868 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:27.154308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:27.168324 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74236910-3b1c-4dcc-8134-926c49bc30ff admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:27.234717 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:27.235422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:27.235715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:27.235993 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] PATH_INFO: `/v3/projects/5731cc4470cf401293268f89e053f514` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:27.238092 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5731cc4470cf401293268f89e053f514)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:27.240046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:27.254255 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:27.410278 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5731cc4470cf401293268f89e053f514', 'request_id': 'req-ba53baca-d2d5-4c01-b5ec-869526afaf4b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.416228 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5731cc4470cf401293268f89e053f514', 'request_id': 'req-ba53baca-d2d5-4c01-b5ec-869526afaf4b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.582185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5731cc4470cf401293268f89e053f514', 'request_id': 'req-ba53baca-d2d5-4c01-b5ec-869526afaf4b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.595687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] The token cache is being invalidate because project 5731cc4470cf401293268f89e053f514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:27.595986 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.596641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.598198 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1388/2754] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:27 2022] DELETE /identity/v3/projects/5731cc4470cf401293268f89e053f514 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:27.602250 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:27.604023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:27.618188 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe047a8-b3be-48cb-9503-a76a9c15b734 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:27.678244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:27.678926 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:27.679211 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:27.679487 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] PATH_INFO: `/v3/users/8a5bf8956207467f95d56333269191a6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:27.680637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5bf8956207467f95d56333269191a6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:27.682555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:27.998533 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5bf8956207467f95d56333269191a6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:27.999507 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5bf8956207467f95d56333269191a6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:28.008830 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:28.014484 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5bf8956207467f95d56333269191a6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:28.170568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5bf8956207467f95d56333269191a6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:28.178757 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-00f00db9-9252-456b-82b2-4511fde248e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5bf8956207467f95d56333269191a6'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:28.184803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1367/2755] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:27 2022] DELETE /identity/v3/users/8a5bf8956207467f95d56333269191a6 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:28.915094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:28.918275 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:28.934374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53baca-d2d5-4c01-b5ec-869526afaf4b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:29.018363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:29.019303 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.019594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.019877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] PATH_INFO: `/v3/projects/4afed1a162ea49468f0e9058ad5d479b` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.021021 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4afed1a162ea49468f0e9058ad5d479b)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:29.022974 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:29.281069 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4afed1a162ea49468f0e9058ad5d479b', 'request_id': 'req-59e82145-360b-4491-8460-8e79cd849d33'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:29.286576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4afed1a162ea49468f0e9058ad5d479b', 'request_id': 'req-59e82145-360b-4491-8460-8e79cd849d33'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:29.461619 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4afed1a162ea49468f0e9058ad5d479b', 'request_id': 'req-59e82145-360b-4491-8460-8e79cd849d33'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:29.475687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] The token cache is being invalidate because project 4afed1a162ea49468f0e9058ad5d479b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:29.475687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:29.477641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-59e82145-360b-4491-8460-8e79cd849d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:29.477641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1389/2756] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:28 2022] DELETE /identity/v3/projects/4afed1a162ea49468f0e9058ad5d479b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:29.483045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.483564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.483724 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.503578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:29.504806 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:29.582974 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1368/2757] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:29.592551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021391f-0642-40da-a713-847076cd88b9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.592947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021391f-0642-40da-a713-847076cd88b9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.593425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021391f-0642-40da-a713-847076cd88b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.621098 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-6021391f-0642-40da-a713-847076cd88b9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:29.622977 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-6021391f-0642-40da-a713-847076cd88b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:29.635631 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1390/2758] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:29.641124 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:29.652952 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91ed7b-8924-4298-ab37-f6c29c4ff64f None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:29.661854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:29.662579 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.663017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.663401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.664252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:25:29.664721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:29.667028 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:29.685193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1369/2759] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:25:29.691736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021391f-0642-40da-a713-847076cd88b9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:29.704823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021391f-0642-40da-a713-847076cd88b9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:29.710331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:29.711046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.711336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.711617 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.712323 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:29.714483 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:29.901594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1391/2760] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:25:29 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:25:29.906160 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:29.920126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537048c-760f-456e-9ba2-02b6a4a9e03c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:29.985809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:29.986460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:29.986813 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:29.987091 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:29.987756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:29.989802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:30.210505 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1370/2761] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:25:29 2022] POST /identity/v3/users => generated 286 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:25:30.214925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:30.220586 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:30.225929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fd9f-8469-4b62-9ea0-55ab42e020a7 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:30.230480 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:30.231112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.231396 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.231666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.232306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:30.234425 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:30.244445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1392/2762] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:30.251542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:30.266548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ce5aa-c451-4459-8af7-664d9321aca9 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:30.271303 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:30.271959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.272237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.272509 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.273150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:30.274798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c51f2-7db9-4c3e-85bf-2be4efbbc423 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:30.285762 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1371/2763] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:25:30.289973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:30.301491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50617d-9a99-4fe6-a9dd-b21a02b21604 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:30.306388 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:30.307114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.307403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.307718 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] PATH_INFO: `/v3/projects/b0123e8d2e614e98af18443fc4a073c6/users/28809717ce904443aa8916c380cfbb46/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.325363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0123e8d2e614e98af18443fc4a073c6, user_id=28809717ce904443aa8916c380cfbb46, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:30.328161 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:30.484149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1393/2764] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:25:30 2022] PUT /identity/v3/projects/b0123e8d2e614e98af18443fc4a073c6/users/28809717ce904443aa8916c380cfbb46/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:30.553229 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.553807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.553807 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.583603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] MFA Rules not processed for user `28809717ce904443aa8916c380cfbb46`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:30.584832 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:30.616022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:30.652244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1372/2765] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:30.665293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:30.667387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:30.739465 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9efefc-c053-4f91-a716-894c8a8c4acd admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:30.744801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:30.745777 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.746265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.746715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.759213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:30.760672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268391f9-dac7-4ea9-a210-e4a0fa7abb54 service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:30.770538 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1394/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:30.798697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:30.865374 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efba31-0b62-415b-87fc-e7e56c7005d5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:30.870445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:30.871117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:30.871397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:30.871703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] PATH_INFO: `/v3/users/28809717ce904443aa8916c380cfbb46` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:30.873017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28809717ce904443aa8916c380cfbb46)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:30.875078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:31.283738 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28809717ce904443aa8916c380cfbb46'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:31.284425 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28809717ce904443aa8916c380cfbb46'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:31.298961 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28809717ce904443aa8916c380cfbb46'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:31.640763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28809717ce904443aa8916c380cfbb46'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:31.651578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28809717ce904443aa8916c380cfbb46'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:31.665095 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1373/2767] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:25:30 2022] DELETE /identity/v3/users/28809717ce904443aa8916c380cfbb46 => generated 0 bytes in 867 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:31.670129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:31.670684 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:31.670884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:31.692317 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:25:31.693725 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:31.770050 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1395/2768] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:25:31.783597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:31.786405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:31.861557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba257-6095-47fb-8875-cacdd1c7a8f5 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:31.865196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:31.865808 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:31.866082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:31.866354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:31.874361 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:31.878180 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:31.879994 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:31.891695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1374/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:32.686884 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:32.688514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:32.703269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4451b8-08aa-4c63-80b9-c8c5b53d1b34 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:32.709973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:32.710855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:32.712121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:32.712878 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] PATH_INFO: `/v3/projects/b0123e8d2e614e98af18443fc4a073c6` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:32.727183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0123e8d2e614e98af18443fc4a073c6)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:32.727183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:32.962911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0123e8d2e614e98af18443fc4a073c6', 'request_id': 'req-7d917430-2424-4016-b25a-6f0be7c218a5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:32.969086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0123e8d2e614e98af18443fc4a073c6', 'request_id': 'req-7d917430-2424-4016-b25a-6f0be7c218a5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:33.284746 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0123e8d2e614e98af18443fc4a073c6', 'request_id': 'req-7d917430-2424-4016-b25a-6f0be7c218a5'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:33.286173 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:33.298265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] The token cache is being invalidate because project b0123e8d2e614e98af18443fc4a073c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:25:33.298562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:33.299269 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-7d917430-2424-4016-b25a-6f0be7c218a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:25:33.300861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1396/2770] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:25:32 2022] DELETE /identity/v3/projects/b0123e8d2e614e98af18443fc4a073c6 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:25:43.358162 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:43.785265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:25:48.520993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:25:48.522399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:48.584743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86227abf-7664-4839-aeb2-10ac2246b58b service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:25:48.649252 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:25:48.649758 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:25:48.650102 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:25:48.650398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:25:48.652348 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:25:48.716723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:25:48.718428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:25:48.727801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1375/2771] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:25:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:25:58.723455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=68 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:00.651817 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430b5b-c940-4de9-9584-cac6ed048d86 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:00.652389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430b5b-c940-4de9-9584-cac6ed048d86 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:00.652389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47430b5b-c940-4de9-9584-cac6ed048d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:00.674288 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-47430b5b-c940-4de9-9584-cac6ed048d86 None None] MFA Rules not processed for user `2f141efe3bb74587b5ba13f0cd5d89db`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:00.675671 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-47430b5b-c940-4de9-9584-cac6ed048d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:00.685013 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1397/2772] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:26:03.349413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:03.357561 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:03.452285 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921d4f2-bf2f-4386-9e75-cd47b985bb87 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:03.457182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:03.457805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:03.458120 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:03.458418 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] PATH_INFO: `/v3/users/da3b758503fd46788a6dd0f1b1d153b8` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:03.459659 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3b758503fd46788a6dd0f1b1d153b8)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:03.461623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:03.765814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3b758503fd46788a6dd0f1b1d153b8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:03.767338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3b758503fd46788a6dd0f1b1d153b8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:03.788163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3b758503fd46788a6dd0f1b1d153b8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:03.940341 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3b758503fd46788a6dd0f1b1d153b8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:03.951454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3b758503fd46788a6dd0f1b1d153b8'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:03.959763 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1376/2773] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:26:03 2022] DELETE /identity/v3/users/da3b758503fd46788a6dd0f1b1d153b8 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:26:03.965270 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:03.966898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:03.967686 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:04.017520 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:04.019252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:04.101012 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1398/2774] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:26:04.111682 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:04.113231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:04.191848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981f699-25a3-4ec1-b6bc-e43cb54af104 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:04.195827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:04.196474 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:04.196770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:04.197208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:04.207548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:04.209032 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:04.219384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1377/2775] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:26:04.798547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:04.800053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:04.818642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815b99e-14d6-44e0-91ef-1c93a8153ad6 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:04.829048 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:04.831141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:04.831437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:04.831715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] PATH_INFO: `/v3/projects/18b43e784f8d4c85bb1d714b6f678b37` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:04.835494 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b43e784f8d4c85bb1d714b6f678b37)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:04.837745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:05.029961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b43e784f8d4c85bb1d714b6f678b37', 'request_id': 'req-ca324ead-8df0-4857-a3c5-c03ffd360759'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:05.039761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b43e784f8d4c85bb1d714b6f678b37', 'request_id': 'req-ca324ead-8df0-4857-a3c5-c03ffd360759'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:05.169998 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b43e784f8d4c85bb1d714b6f678b37', 'request_id': 'req-ca324ead-8df0-4857-a3c5-c03ffd360759'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:05.185955 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] The token cache is being invalidate because project 18b43e784f8d4c85bb1d714b6f678b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:26:05.186309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:05.186805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:05.188305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1399/2776] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:26:04 2022] DELETE /identity/v3/projects/18b43e784f8d4c85bb1d714b6f678b37 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:26:10.454667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:10.459052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:10.476411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff94b8-c925-4b14-931c-d3d410f17053 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:10.501921 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:10.538597 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:10.539148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:10.539539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:10.539702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:10.541497 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:10.618048 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:10.624455 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:10.636549 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1378/2777] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:26:15.182395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:15.406594 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:15.411588 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:15.425114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca324ead-8df0-4857-a3c5-c03ffd360759 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:15.431346 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:15.432028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:15.432314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:15.432602 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] PATH_INFO: `/v3/users/660c6df77d0a4750aac30345d6768526` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:15.433823 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=660c6df77d0a4750aac30345d6768526)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:15.436079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:15.660630 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660c6df77d0a4750aac30345d6768526'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:15.661315 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660c6df77d0a4750aac30345d6768526'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:15.675877 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660c6df77d0a4750aac30345d6768526'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:15.895637 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660c6df77d0a4750aac30345d6768526'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:15.905391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8b9efeb5-2285-4b25-b2f6-a4c9f71b8113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660c6df77d0a4750aac30345d6768526'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:15.912804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1400/2778] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:26:15 2022] DELETE /identity/v3/users/660c6df77d0a4750aac30345d6768526 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:26:16.238744 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:16.250201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4556ece-1f7c-43a6-8b0b-bd568d445fbd service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:16.338511 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:16.339181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:16.339458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:16.339733 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] PATH_INFO: `/v3/projects/f7e0c375a58842e18a4306779fd6e167` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:16.341460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e0c375a58842e18a4306779fd6e167)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:16.343865 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:16.519521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e0c375a58842e18a4306779fd6e167', 'request_id': 'req-d731ab29-6d47-4ac6-864b-816336d80fbb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:16.667249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e0c375a58842e18a4306779fd6e167', 'request_id': 'req-d731ab29-6d47-4ac6-864b-816336d80fbb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:16.674020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e0c375a58842e18a4306779fd6e167', 'request_id': 'req-d731ab29-6d47-4ac6-864b-816336d80fbb'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:16.688299 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] The token cache is being invalidate because project f7e0c375a58842e18a4306779fd6e167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:26:16.688782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:16.689620 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-d731ab29-6d47-4ac6-864b-816336d80fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:26:16.692209 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1379/2779] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:26:16 2022] DELETE /identity/v3/projects/f7e0c375a58842e18a4306779fd6e167 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:26:16.699035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:16.699592 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:16.700090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:16.725175 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:16.726404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:16.807899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1401/2780] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:26:16.822539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:16.829794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:16.829794 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:16.838248 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:16.862083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:16.863928 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:16.873925 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1380/2781] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:26:16.878833 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:16.890153 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540029c3-e56e-4997-8ae8-cb60a3c80ef7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:16.909932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:16.909932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:16.909932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:16.909932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:16.910534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:26:16.910534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:16.910534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:16.929595 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1402/2782] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:26:16.934138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:16.944541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f927-e878-483a-8a3a-9c6ce171c776 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:16.949798 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:16.951277 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:16.951584 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:16.951913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:16.952568 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:16.954510 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.133530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1381/2783] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:26:16 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:26:17.137978 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:17.148557 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a284e-a9d5-4aa8-8439-4731fef7ec30 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:17.249804 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:17.251551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.252047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.252479 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.253688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:17.258786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.267491 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:17.441727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1403/2784] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:26:17 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:26:17.447523 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:17.457175 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3d5bf-2238-48ee-884b-a020b705cc78 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:17.462923 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:17.463883 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.464294 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.464639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.467311 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:17.469071 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.480145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1382/2785] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:26:17.484139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:17.498854 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86760196-3a1f-4cf4-bcc9-247044ad17ec admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:17.508329 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:17.509129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.509348 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.509909 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.510528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:17.512440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87f26b-d113-4979-99ab-71261a58e654 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.526561 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1404/2786] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:26:17.530853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:17.541266 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5217f-0bc3-4aa7-a084-8c1a0e1b3f62 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:17.547022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:17.547705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.547984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.548267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] PATH_INFO: `/v3/projects/c070cba92de641518c86248b1e27daea/users/38f279301f8c4c76945d361f19b686ac/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.565078 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c070cba92de641518c86248b1e27daea, user_id=38f279301f8c4c76945d361f19b686ac, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:17.572241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.714573 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1383/2787] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:26:17 2022] PUT /identity/v3/projects/c070cba92de641518c86248b1e27daea/users/38f279301f8c4c76945d361f19b686ac/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:26:17.717218 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.718178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.718178 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.739634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:17.741031 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:17.814213 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1405/2788] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:26:17.826065 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:17.828526 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:17.897785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef379a2f-8b21-411c-bb37-83e07e97ad0b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:17.901637 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:17.902238 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:17.902520 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:17.902821 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:17.916548 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:17.917997 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10e9d7-eec2-4052-85dd-41cd7f621a02 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:17.928183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1384/2789] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:26:25.124966 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:25.127369 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:25.141047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5475b-10d8-46b6-b146-9580d459d739 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:25.145449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:25.146554 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:25.146896 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:25.147121 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:25.149109 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:25.247518 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:25.248368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:25.248863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:25.257514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:25.259664 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:25.274738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1406/2790] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:26:25.332524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] MFA Rules not processed for user `38f279301f8c4c76945d361f19b686ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:25.332524 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:25.550163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1385/2791] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:26:25.557958 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:25.560422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:25.581539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886880f1-78ba-4704-b745-67f2958851ef service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:25.589982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:25.590843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:25.591226 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:25.591855 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:25.622325 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:25.628046 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:25.641983 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1407/2792] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:26:26.852010 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:26.865840 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841d7be-0dd0-4953-a3cc-91e5460fee19 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:26.873805 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:26.874105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:26.874565 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:26.875061 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:26.878791 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:26.898050 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:26.899756 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f712675-71a2-4b0b-9c67-b0f6a8292522 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:26.913478 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1386/2793] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:26:29.776517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:26:29.792058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642038b-a99e-4d2d-a75f-5a323eb7b6ce service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:26:29.796649 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:26:29.797666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:29.797952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:29.798239 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:29.800510 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:29.863681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:29.869322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:26:29.876664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:26:29.876664 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:26:29.882122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:26:29.883687 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7bf33-d158-4070-a1e3-26955fd2b87e service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:26:29.890868 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] MFA Rules not processed for user `38f279301f8c4c76945d361f19b686ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:26:29.892105 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:26:29.899546 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1387/2794] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:26:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:26:29.900307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1408/2795] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:26:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:26:39.885704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=93,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:26:41.112237 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=7 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:27:39.280854 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:27:39.287280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a19a5-0848-4019-98c6-a5e99adfdb21 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:27:39.291093 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:27:39.291603 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:27:39.292421 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:27:39.293388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:27:39.294742 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:39.305863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:27:39.309249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:27:39.317877 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1388/2796] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:27:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:27:41.651196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:27:41.651196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:27:41.651196 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:27:41.658556 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] MFA Rules not processed for user `38f279301f8c4c76945d361f19b686ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:27:41.659841 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:41.668380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1409/2797] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:27:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:27:42.581073 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:27:42.583521 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:42.678638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b30dd-11d6-4234-8370-86d9ab915cd7 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:27:42.688761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:27:42.691097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:27:42.691097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:27:42.691097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:27:42.693205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:42.714400 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:27:42.716685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:27:42.728016 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1389/2798] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:27:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:27:48.544223 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:27:48.549726 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56728724-8a69-4334-9f7c-9991c7c82ae3 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:27:48.556420 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:27:48.557790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:27:48.557790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:27:48.558591 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:27:48.560264 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:48.574336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:27:48.576079 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:27:48.587503 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1410/2799] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:27:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:27:52.722940 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=38 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:27:55.876140 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:27:55.878852 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:55.903902 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ef06-3fce-46c3-b6ba-f14ff224fb70 service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:27:55.910646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:27:55.911269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:27:55.911540 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:27:55.911812 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:27:55.913646 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:27:55.934941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:27:55.939337 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:27:55.955875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1390/2800] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:27:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:27:58.582573 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=9 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:01.477842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:01.496504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a20bbd-cdea-4779-9ad3-391804c56327 service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:01.520979 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dd40d8b3cb543feab8424f1d87b7126', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:01.521814 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:01.522258 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:01.522743 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:01.524908 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:01.558053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:01.571925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:01.589149 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1411/2801] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:28:05.949163 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:09.281833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:09.289867 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c666d9c-9927-40c9-b780-7d8cb3f7f23a service nova] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:09.294257 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:09.294893 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:09.295173 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:09.295446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:09.297498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:09.311295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:09.312853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:09.323708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1391/2802] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:28:11.568280 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=6 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:18.022351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:18.040309 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f45f7-d2ce-42cc-b713-abfb01c444a1 service nova] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:18.049597 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:18.050242 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:18.050504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:18.051183 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:18.053713 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:18.070274 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:18.072078 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:18.085091 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1412/2803] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:28:18.279258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:18.279258 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:18.291245 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75932b-3d81-45d2-9435-22325417003b service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:18.295901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:18.296531 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:18.296810 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:18.297080 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] PATH_INFO: `/v3/users/a9ebeec374454468aae1d2b5fde21648` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:18.298182 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ebeec374454468aae1d2b5fde21648)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:18.300117 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:18.813001 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ebeec374454468aae1d2b5fde21648'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:18.813685 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ebeec374454468aae1d2b5fde21648'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:18.826205 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ebeec374454468aae1d2b5fde21648'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.033024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ebeec374454468aae1d2b5fde21648'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.040653 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ebeec374454468aae1d2b5fde21648'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.047284 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1392/2804] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:28:18 2022] DELETE /identity/v3/users/a9ebeec374454468aae1d2b5fde21648 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:19.423610 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:19.442600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7975d-85f3-4c81-9de7-2b89a2e58d95 service placement] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:19.532612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:19.533247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:19.533534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:19.533810 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] PATH_INFO: `/v3/projects/52e8949a7d05434396f8e8631f2cdb72` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:19.535000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e8949a7d05434396f8e8631f2cdb72)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:19.536920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:19.726102 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e8949a7d05434396f8e8631f2cdb72', 'request_id': 'req-262eb917-30e3-40d4-9999-06c5a36f515b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.733338 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e8949a7d05434396f8e8631f2cdb72', 'request_id': 'req-262eb917-30e3-40d4-9999-06c5a36f515b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.882644 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e8949a7d05434396f8e8631f2cdb72', 'request_id': 'req-262eb917-30e3-40d4-9999-06c5a36f515b'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.899735 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] The token cache is being invalidate because project 52e8949a7d05434396f8e8631f2cdb72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:28:19.900385 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.901234 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:19.903034 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1413/2805] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:28:19 2022] DELETE /identity/v3/projects/52e8949a7d05434396f8e8631f2cdb72 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:19.907667 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:19.909197 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:19.925552 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f61b6-dc69-4ca8-8ef8-240856b1a656 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:19.997542 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:19.998092 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:19.998367 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:19.998662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] PATH_INFO: `/v3/users/415da9e74d4a495b907572f9131e84c0` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:19.999802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] RBAC: Authorizing `identity:delete_user(user_id=415da9e74d4a495b907572f9131e84c0)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:20.001706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:20.400892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415da9e74d4a495b907572f9131e84c0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:20.400892 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415da9e74d4a495b907572f9131e84c0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:20.410452 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:20.411208 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:20.412705 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:20.412848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415da9e74d4a495b907572f9131e84c0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:20.484731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb917-30e3-40d4-9999-06c5a36f515b admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:20.488454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:20.489076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:20.489360 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:20.489685 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:20.491399 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:20.503186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:20.504616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:20.514688 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1414/2806] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 06:28:20.581358 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415da9e74d4a495b907572f9131e84c0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:20.597097 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415da9e74d4a495b907572f9131e84c0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:20.605274 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1393/2807] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:28:19 2022] DELETE /identity/v3/users/415da9e74d4a495b907572f9131e84c0 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:20.982968 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:20.984967 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:20.999236 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02fc83-1924-4556-8e31-b28f7f6a32e0 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:21.028612 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:21.079459 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:21.092583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:21.092583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:21.092583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] PATH_INFO: `/v3/projects/03e76ed17eff49e289513ca492f561b9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:21.092583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e76ed17eff49e289513ca492f561b9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:21.092583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:21.266473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e76ed17eff49e289513ca492f561b9', 'request_id': 'req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:21.271990 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e76ed17eff49e289513ca492f561b9', 'request_id': 'req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:21.393026 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e76ed17eff49e289513ca492f561b9', 'request_id': 'req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:21.405874 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] The token cache is being invalidate because project 03e76ed17eff49e289513ca492f561b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:28:21.406195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:21.406894 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:21.408413 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1415/2808] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:28:20 2022] DELETE /identity/v3/projects/03e76ed17eff49e289513ca492f561b9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:24.900590 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:24.902290 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:24.991287 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e3bd-a7b7-4c03-8df8-d09008e2372a admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:25.057843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:25.062954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:25.062954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:25.062954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:25.062954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:25.075037 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:25.076833 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:25.088174 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1394/2809] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:28:29.528856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:29.534898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:29.601441 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ddb3b-9bc4-46e8-84ea-52cba31f5c92 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:29.607514 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:29.608158 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:29.608437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:29.608736 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] PATH_INFO: `/v3/users/2f141efe3bb74587b5ba13f0cd5d89db` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:29.609900 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f141efe3bb74587b5ba13f0cd5d89db)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:29.611885 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:29.942549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f141efe3bb74587b5ba13f0cd5d89db'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:29.943504 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f141efe3bb74587b5ba13f0cd5d89db'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:29.948703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:29.957773 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f141efe3bb74587b5ba13f0cd5d89db'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.084306 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f141efe3bb74587b5ba13f0cd5d89db'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.092738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f141efe3bb74587b5ba13f0cd5d89db'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.099616 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1416/2810] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:28:29 2022] DELETE /identity/v3/users/2f141efe3bb74587b5ba13f0cd5d89db => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:30.450696 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:30.469399 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5360f-fa22-41c4-a566-00d8fc0175c5 service placement] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:30.543029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:30.543746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:30.544024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:30.544297 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] PATH_INFO: `/v3/projects/fde0e661bd5646bab91b5e6b623a2e7e` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:30.545446 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde0e661bd5646bab91b5e6b623a2e7e)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:30.547419 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:30.561255 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:30.731569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde0e661bd5646bab91b5e6b623a2e7e', 'request_id': 'req-13676323-faee-4ad7-84f6-a55528af7525'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.860993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde0e661bd5646bab91b5e6b623a2e7e', 'request_id': 'req-13676323-faee-4ad7-84f6-a55528af7525'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.867827 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde0e661bd5646bab91b5e6b623a2e7e', 'request_id': 'req-13676323-faee-4ad7-84f6-a55528af7525'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.883629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] The token cache is being invalidate because project fde0e661bd5646bab91b5e6b623a2e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:28:30.884082 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.885539 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:30.887101 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1395/2811] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:28:30 2022] DELETE /identity/v3/projects/fde0e661bd5646bab91b5e6b623a2e7e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:30.891627 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:30.893738 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:30.915184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab751ed-1307-4ed8-b8fd-ea5ebe748eaa admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:30.977727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:30.979525 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:30.980281 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:30.980575 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] PATH_INFO: `/v3/users/0241c31fc7a0437cbe997b30cbc90fcf` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:30.982202 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0241c31fc7a0437cbe997b30cbc90fcf)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:30.984277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:31.287409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0241c31fc7a0437cbe997b30cbc90fcf'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:31.288141 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0241c31fc7a0437cbe997b30cbc90fcf'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:31.303047 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0241c31fc7a0437cbe997b30cbc90fcf'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:31.402558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0241c31fc7a0437cbe997b30cbc90fcf'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:31.410606 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-0da0f6b6-8f84-40e2-889f-868f6c87a0f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0241c31fc7a0437cbe997b30cbc90fcf'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:31.417750 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1417/2812] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:28:30 2022] DELETE /identity/v3/users/0241c31fc7a0437cbe997b30cbc90fcf => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:28:31.785241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:31.787853 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:31.808122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676323-faee-4ad7-84f6-a55528af7525 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:31.877719 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:31.878388 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:31.878681 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:31.878962 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] PATH_INFO: `/v3/projects/ec0cd181b35644639e7b0c20e865c6b6` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:31.881574 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0cd181b35644639e7b0c20e865c6b6)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:31.883630 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:32.029594 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0cd181b35644639e7b0c20e865c6b6', 'request_id': 'req-70aa4893-2be5-4e0e-929d-18207c0bd810'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:32.152057 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0cd181b35644639e7b0c20e865c6b6', 'request_id': 'req-70aa4893-2be5-4e0e-929d-18207c0bd810'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:32.159153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0cd181b35644639e7b0c20e865c6b6', 'request_id': 'req-70aa4893-2be5-4e0e-929d-18207c0bd810'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:32.173500 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] The token cache is being invalidate because project ec0cd181b35644639e7b0c20e865c6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:28:32.173843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:32.174608 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:28:32.176361 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1396/2813] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:28:31 2022] DELETE /identity/v3/projects/ec0cd181b35644639e7b0c20e865c6b6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:28:41.591094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=13,keystone:SELECT=63,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:42.169783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=70,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:28:56.361367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:56.364295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:56.364295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:56.384780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] MFA Rules not processed for user `38f279301f8c4c76945d361f19b686ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:28:56.386703 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:56.451758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1418/2814] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:28:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:28:56.782851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:28:56.784331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:56.848787 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa4893-2be5-4e0e-929d-18207c0bd810 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:28:56.853089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:28:56.853714 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:28:56.853991 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:28:56.854264 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:28:56.856090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:28:56.866376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:28:56.867880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238344-980e-42a7-b06b-6bb806edc03c service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:28:56.876139 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1397/2815] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:28:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:29:06.444572 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=25 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:29:06.872547 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=26 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:29:42.265583 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:29:42.267376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:29:42.368241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc733-4115-4202-b1ed-810af0bdbd5c None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:29:42.376848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:29:42.377880 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:29:42.378368 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:29:42.378890 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:29:42.381609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:29:42.402585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:29:42.405292 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:29:42.430071 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1419/2816] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:29:52.412410 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=32 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] MFA Rules not processed for user `38f279301f8c4c76945d361f19b686ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:30:15.936785 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1398/2817] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:30:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:30:16.250727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:30:16.252107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:30:16.263940 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b99005-104a-44ee-96c4-0a77bc1ab01d service placement] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:30:16.267490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:30:16.268089 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:30:16.268367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:30:16.268640 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:30:16.270394 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:30:16.280390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:30:16.281811 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:30:16.290715 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1420/2818] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:30:25.915695 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:30:26.286265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=6 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:25.432177 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:25.433655 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:25.496223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7edd83-3feb-486a-bb96-2d724c3af36e None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:25.501958 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:25.502578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:25.502889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:25.503167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:25.505354 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:25.517114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:25.518639 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:25.528557 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1399/2819] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:27.173112 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:27.183075 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b98ce-8340-469e-9774-f97ba936e96d service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:27.187286 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:27.188019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:27.188310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:27.188624 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:27.190562 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:27.264276 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:27.265729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:27.276395 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1421/2820] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:32.287345 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:32.288801 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:32.304957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f72ec-ff1b-4e50-a89f-c6b97a529a57 service cinder] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:32.309834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:32.310465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:32.310783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:32.311066 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] PATH_INFO: `/v3/users/38f279301f8c4c76945d361f19b686ac` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:32.312204 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f279301f8c4c76945d361f19b686ac)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:32.314130 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:32.744286 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f279301f8c4c76945d361f19b686ac'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:32.744957 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f279301f8c4c76945d361f19b686ac'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:32.756834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f279301f8c4c76945d361f19b686ac'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:33.102269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f279301f8c4c76945d361f19b686ac'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:33.110024 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-3a2a518f-be20-4279-b840-2d5a03c7bc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f279301f8c4c76945d361f19b686ac'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:33.117159 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1400/2821] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:32 2022] DELETE /identity/v3/users/38f279301f8c4c76945d361f19b686ac => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:33.623973 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:33.636305 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13ee52-6a4e-43a3-983c-c6d353bdc6fd service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:33.699672 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:33.700232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:33.700517 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:33.700794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] PATH_INFO: `/v3/projects/c070cba92de641518c86248b1e27daea` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:33.701927 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c070cba92de641518c86248b1e27daea)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:33.703917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:34.051390 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c070cba92de641518c86248b1e27daea', 'request_id': 'req-ced31de6-91e0-49d3-9121-9e32a1c32fce'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:34.059096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c070cba92de641518c86248b1e27daea', 'request_id': 'req-ced31de6-91e0-49d3-9121-9e32a1c32fce'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:34.341982 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c070cba92de641518c86248b1e27daea', 'request_id': 'req-ced31de6-91e0-49d3-9121-9e32a1c32fce'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:34.359064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] The token cache is being invalidate because project c070cba92de641518c86248b1e27daea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:34.359529 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:34.360114 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-ced31de6-91e0-49d3-9121-9e32a1c32fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:34.361638 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1422/2822] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:33 2022] DELETE /identity/v3/projects/c070cba92de641518c86248b1e27daea => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:34.367494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:34.368391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:34.368391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:34.389440 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:34.390739 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:34.459857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1401/2823] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:31:34.469769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:34.470133 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:34.470639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:34.509136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:34.510403 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:34.518646 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1423/2824] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:31:34.525782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:34.536467 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea246ba7-1f67-4fd1-a5fc-c83fea3c4e3c None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:34.543502 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:34.544217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:34.544498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:34.544771 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:34.545434 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:31:34.545735 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:34.547555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:34.562146 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1402/2825] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:31:34.569085 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:34.579073 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc58762-2372-4fcd-86d2-5cd1c51d98de None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:34.584901 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:34.585543 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:34.585839 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:34.586507 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:34.587189 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:34.592720 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:34.611445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:34.751054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1424/2826] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:34 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:34.756917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:34.762850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:34.769730 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf339e4-9be3-495a-9c39-9d7d538333c7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:34.838494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:34.839132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:34.839411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:34.839686 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:34.840307 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:34.842640 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:34.999968 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1403/2827] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:34 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:35.007903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.018016 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cee25-a412-4114-a0b2-426696311449 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.025393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.026081 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.026364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.026695 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.027393 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.029019 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.041551 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1425/2828] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:35.079815 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.090376 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312ca16-0768-43b0-9a92-8e7f96b2fb4c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.095445 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.096133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.096618 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.097151 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.098306 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.099972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c3776-f2f0-4931-84e8-b810afdb734e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.111972 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1404/2829] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:35.117883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.131714 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae858-c261-4cb1-89f7-c61fd69c0826 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.136534 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.137452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.137864 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.138267 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] PATH_INFO: `/v3/projects/cca3b155db894cc4a5da711ddf7cc8d3/users/bcc617263d444894900a66ce40f6bbd9/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.156419 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca3b155db894cc4a5da711ddf7cc8d3, user_id=bcc617263d444894900a66ce40f6bbd9, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.160145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.286907 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1426/2830] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:35 2022] PUT /identity/v3/projects/cca3b155db894cc4a5da711ddf7cc8d3/users/bcc617263d444894900a66ce40f6bbd9/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:35.299465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.299828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.300315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.336423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] MFA Rules not processed for user `bcc617263d444894900a66ce40f6bbd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:35.338330 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:35.408569 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1405/2831] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:35.414970 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.487035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b03b-87bf-45ad-89e9-ede438af7f07 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.493574 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.494232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.494527 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.494861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.495566 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.497501 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.691787 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1427/2832] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:35 2022] POST /identity/v3/projects => generated 367 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:31:35.696134 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.707132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c68fb-73c9-4975-b39a-51c1409515f5 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.770477 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.771116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.771391 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.771663 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.772302 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.774750 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.954401 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:35.961625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1406/2833] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:35 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:31:35.968917 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:35.979918 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad6cddb-20e9-4ebb-b898-a4c5a5f4fcc9 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:35.984442 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:35.985038 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:35.985322 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:35.985596 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:35.986241 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:35.987992 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:35.998482 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1428/2834] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:36.002731 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:36.013428 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7ddd-7b06-4c49-b786-11df5585839b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:36.018725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:36.019423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:36.019706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:36.019984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] PATH_INFO: `/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756/users/12324069962c420e89e2dfe5a08d3ff4/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:36.037463 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b0584b27cf4e88b76fa6929a3ad756, user_id=12324069962c420e89e2dfe5a08d3ff4, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:36.040126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:36.247690 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1407/2835] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:36 2022] PUT /identity/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756/users/12324069962c420e89e2dfe5a08d3ff4/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:36.251856 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:36.263266 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71836-a3bc-4e4a-8177-fb08e2e2257e admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:36.332709 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:36.335035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:36.335035 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:36.335593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:36.335919 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:36.339053 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30dee-5bc3-4f5f-b9b7-d69a48936638 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:36.352674 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1429/2836] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:36.357450 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:36.373335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fea2a2-d836-4cf4-8c6d-c17f81620afe admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:36.378158 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:36.378855 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:36.379138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:36.379408 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] PATH_INFO: `/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756/users/12324069962c420e89e2dfe5a08d3ff4/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:36.382033 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b0584b27cf4e88b76fa6929a3ad756, user_id=12324069962c420e89e2dfe5a08d3ff4, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:36.384759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:36.397218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1408/2837] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:36 2022] PUT /identity/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756/users/12324069962c420e89e2dfe5a08d3ff4/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:31:36.420539 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:36.420826 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:36.421152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:36.453668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] MFA Rules not processed for user `12324069962c420e89e2dfe5a08d3ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:36.454960 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:36.464861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:36.544558 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1430/2838] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:31:36.556385 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:36.558168 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:36.616723 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90bde5-aef0-497f-8db4-993c9c1f9211 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:36.620424 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:36.621052 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:36.621331 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:36.621604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:36.631570 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:36.632977 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66726f42-c023-43f9-b4c9-431107e10d2a service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:36.642454 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1409/2839] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:41.525179 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:41.592080 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2108db-b926-44f5-8a75-f49d51fb4b7b None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:41.596760 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:41.597400 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:41.597681 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:41.597956 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] PATH_INFO: `/v3/users/bcc617263d444894900a66ce40f6bbd9` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:41.599797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc617263d444894900a66ce40f6bbd9)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:41.601759 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:41.939271 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc617263d444894900a66ce40f6bbd9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:41.939948 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc617263d444894900a66ce40f6bbd9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:41.953899 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc617263d444894900a66ce40f6bbd9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:42.113601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc617263d444894900a66ce40f6bbd9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:42.122105 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc617263d444894900a66ce40f6bbd9'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:42.129441 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1431/2840] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:41 2022] DELETE /identity/v3/users/bcc617263d444894900a66ce40f6bbd9 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:42.134322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:42.134967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:42.134967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:42.154223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:42.155494 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:42.228167 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1410/2841] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:42.240361 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:42.241790 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:42.275253 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:42.305609 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bea419-2570-4946-85d4-fe0ff74fc5a1 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:42.313601 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:42.314137 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:42.314416 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:42.314722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:42.326210 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:42.327710 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:42.339252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1432/2842] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:43.074905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:43.076333 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:43.089980 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f71ad3-5379-4ebb-a772-01c63cfc1beb None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:43.094692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:43.095411 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:43.095765 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:43.096041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] PATH_INFO: `/v3/projects/cca3b155db894cc4a5da711ddf7cc8d3` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:43.097235 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca3b155db894cc4a5da711ddf7cc8d3)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:43.100169 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:43.308468 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca3b155db894cc4a5da711ddf7cc8d3', 'request_id': 'req-1077fa77-23d5-4aa9-aaf1-84175a2490cd'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.412315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca3b155db894cc4a5da711ddf7cc8d3', 'request_id': 'req-1077fa77-23d5-4aa9-aaf1-84175a2490cd'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.417008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:43.419322 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca3b155db894cc4a5da711ddf7cc8d3', 'request_id': 'req-1077fa77-23d5-4aa9-aaf1-84175a2490cd'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.433208 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] The token cache is being invalidate because project cca3b155db894cc4a5da711ddf7cc8d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:43.433513 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.434207 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.436123 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1411/2843] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:43 2022] DELETE /identity/v3/projects/cca3b155db894cc4a5da711ddf7cc8d3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:31:43.439949 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:43.451257 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbfba6-7ea8-4d00-81c8-625269a2b8cd service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:43.518512 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:43.519180 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:43.519464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:43.519745 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] PATH_INFO: `/v3/users/12324069962c420e89e2dfe5a08d3ff4` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:43.521545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12324069962c420e89e2dfe5a08d3ff4)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:43.523580 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:43.839570 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12324069962c420e89e2dfe5a08d3ff4'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.840187 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12324069962c420e89e2dfe5a08d3ff4'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.854167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12324069962c420e89e2dfe5a08d3ff4'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.981473 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12324069962c420e89e2dfe5a08d3ff4'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.989666 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-803a4b44-c2aa-47ac-92b2-14645c192187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12324069962c420e89e2dfe5a08d3ff4'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:43.995696 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1433/2844] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:43 2022] DELETE /identity/v3/users/12324069962c420e89e2dfe5a08d3ff4 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:44.627876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:44.627876 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:44.646954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077fa77-23d5-4aa9-aaf1-84175a2490cd admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:44.712458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:44.712458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:44.712458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:44.712458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] PATH_INFO: `/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:44.719864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b0584b27cf4e88b76fa6929a3ad756)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:44.719864 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:44.920668 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b0584b27cf4e88b76fa6929a3ad756', 'request_id': 'req-8a7a64aa-3219-4230-89ab-868f4c40f455'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:45.073370 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b0584b27cf4e88b76fa6929a3ad756', 'request_id': 'req-8a7a64aa-3219-4230-89ab-868f4c40f455'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:45.079834 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b0584b27cf4e88b76fa6929a3ad756', 'request_id': 'req-8a7a64aa-3219-4230-89ab-868f4c40f455'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:45.097366 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] The token cache is being invalidate because project f2b0584b27cf4e88b76fa6929a3ad756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:45.097803 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:45.098790 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-8a7a64aa-3219-4230-89ab-868f4c40f455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:45.101325 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1412/2845] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:44 2022] DELETE /identity/v3/projects/f2b0584b27cf4e88b76fa6929a3ad756 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:31:45.110476 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43184529-5002-4205-b075-26a8463dd929 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.111059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43184529-5002-4205-b075-26a8463dd929 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.111059 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43184529-5002-4205-b075-26a8463dd929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.134505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-43184529-5002-4205-b075-26a8463dd929 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:45.135805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-43184529-5002-4205-b075-26a8463dd929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:45.189793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:45.213374 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1434/2846] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 06:31:45.221201 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.221491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.221770 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.241851 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:45.243114 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:45.251083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1413/2847] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:45.256014 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43184529-5002-4205-b075-26a8463dd929 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.265665 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43184529-5002-4205-b075-26a8463dd929 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.270422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.271003 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.271291 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.271568 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.272216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:31:45.272528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.274128 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.285740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1435/2848] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:31:45.290111 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.299351 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345eae12-91e3-4f15-ab85-9914a62baa6d None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.305017 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.305616 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.305901 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.306187 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.306846 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.308761 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.504777 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1414/2849] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:45 2022] POST /identity/v3/projects => generated 389 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:45.509076 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.519675 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ce5a-4163-4e54-a3fd-25affb06b233 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.592321 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.592945 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.593232 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.593511 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.594148 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.596184 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.788882 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1436/2850] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:45 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:45.793087 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.804629 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df3dc6-9d1e-406c-bbdf-41d2ccf9364e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.809222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.809828 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.810109 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.810384 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.811067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.812746 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.823029 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1415/2851] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:45.827252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.837828 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614e50b-338d-4962-9374-831356f59398 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.842475 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.843123 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.843392 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.843668 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.844310 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.845921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7f9c1-ed0f-4fa7-b895-a9705de5fcea admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.857694 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1437/2852] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:45.862054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:45.874544 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cf636-7bf1-4ca6-a343-a6255d5d0de8 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:45.879202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:45.880153 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.880498 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.880780 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] PATH_INFO: `/v3/projects/761548f6db7e49e8a858ab97d77fbf90/users/5fb0893ea56843449249aa094708ee40/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:45.896819 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=761548f6db7e49e8a858ab97d77fbf90, user_id=5fb0893ea56843449249aa094708ee40, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:45.899517 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:45.987506 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1416/2853] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:45 2022] PUT /identity/v3/projects/761548f6db7e49e8a858ab97d77fbf90/users/5fb0893ea56843449249aa094708ee40/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:45.998545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:45.998872 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:45.999152 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.027921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] MFA Rules not processed for user `5fb0893ea56843449249aa094708ee40`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:46.029136 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:46.099017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1438/2854] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:46.103772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:46.123592 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:46.168959 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69f6f-0586-4dab-b74a-27baef036eee admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:46.173709 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:46.174338 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:46.174596 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:46.174911 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.175550 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:46.177619 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:46.314782 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1417/2855] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:46 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 06:31:46.319314 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:46.330272 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578b4a3-6b36-4b92-bb07-71875a260953 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:46.373593 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:46.389018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:46.389629 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:46.389911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:46.390186 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.390848 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:46.392926 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:46.530920 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1439/2856] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:46 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:46.534978 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:46.546054 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b51bd7-8aff-4611-9689-a62cd2f794b8 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:46.550985 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:46.551720 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:46.552008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:46.552282 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.553112 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:46.555045 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:46.567623 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1418/2857] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:46.572549 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:46.583375 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f661a8-cd23-40dc-b3ef-7e5afd86dc2a admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:46.589559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:46.590247 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:46.591265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:46.591265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] PATH_INFO: `/v3/projects/3a533d25f80d46ffae960c242c30c530/users/17fc9fa751164e2ca6062a7e3e49ed73/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.608679 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a533d25f80d46ffae960c242c30c530, user_id=17fc9fa751164e2ca6062a7e3e49ed73, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:46.611490 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:46.925626 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1440/2858] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:46 2022] PUT /identity/v3/projects/3a533d25f80d46ffae960c242c30c530/users/17fc9fa751164e2ca6062a7e3e49ed73/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:46.930041 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:46.940136 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580219-44c9-411d-911f-3298ad85ae3c admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:46.996649 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:46.997433 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:46.997743 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:46.998020 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:46.998722 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:47.000327 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4159f90-c915-4736-9326-ebc704318aea admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:47.010423 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1419/2859] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:47.015061 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:47.026711 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439a456-506a-4a31-b677-5e45dda1edcc admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:47.032100 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:47.032758 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:47.033042 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:47.033353 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] PATH_INFO: `/v3/projects/3a533d25f80d46ffae960c242c30c530/users/17fc9fa751164e2ca6062a7e3e49ed73/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:47.035277 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a533d25f80d46ffae960c242c30c530, user_id=17fc9fa751164e2ca6062a7e3e49ed73, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:47.038925 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:47.051351 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1441/2860] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:47 2022] PUT /identity/v3/projects/3a533d25f80d46ffae960c242c30c530/users/17fc9fa751164e2ca6062a7e3e49ed73/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:47.062530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:47.062843 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:47.063122 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:47.093698 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] MFA Rules not processed for user `17fc9fa751164e2ca6062a7e3e49ed73`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:47.095447 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:47.178397 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1420/2861] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:47.189727 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:47.191524 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:47.257961 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2085f-2730-4b12-800d-acdfa1d0e586 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:47.261806 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:47.262438 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:47.262754 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:47.263036 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:47.275190 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:47.276670 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f83f-5710-4b58-bbee-d85a14ac90fa service cinder] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:47.288096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1442/2862] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:47.434530 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:47.447875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:47.499007 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea43c3-f004-4062-972b-bb1712bbb3e9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:47.503657 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:47.504850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:47.505171 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:47.505451 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] PATH_INFO: `/v3/users/5fb0893ea56843449249aa094708ee40` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:47.507099 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb0893ea56843449249aa094708ee40)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:47.508973 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:47.818601 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb0893ea56843449249aa094708ee40'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:47.819268 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb0893ea56843449249aa094708ee40'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:47.831019 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb0893ea56843449249aa094708ee40'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:47.976013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb0893ea56843449249aa094708ee40'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:47.983635 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb0893ea56843449249aa094708ee40'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:47.990223 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1421/2863] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:47 2022] DELETE /identity/v3/users/5fb0893ea56843449249aa094708ee40 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:47.994903 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:47.995364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:47.995364 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:48.014763 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:48.015963 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:48.075449 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1443/2864] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:48.086473 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:48.087983 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:48.147604 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdb81c-c6c4-4bcb-84a3-8f99146f350e admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:48.151253 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:48.151941 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:48.152217 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:48.152489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:48.161732 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:48.163218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:48.172704 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1422/2865] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:48.938440 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:48.940017 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:48.954560 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f739b-d21b-4e17-a9d4-6d8aa75976a9 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:48.959912 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:48.960553 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:48.961380 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:48.961933 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] PATH_INFO: `/v3/projects/761548f6db7e49e8a858ab97d77fbf90` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:48.963295 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] RBAC: Authorizing `identity:delete_project(project_id=761548f6db7e49e8a858ab97d77fbf90)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:48.965215 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:49.218753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:49.230972 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '761548f6db7e49e8a858ab97d77fbf90', 'request_id': 'req-c42ccc78-735d-43fb-8305-c706637c677c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:49.237643 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '761548f6db7e49e8a858ab97d77fbf90', 'request_id': 'req-c42ccc78-735d-43fb-8305-c706637c677c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:49.708000 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '761548f6db7e49e8a858ab97d77fbf90', 'request_id': 'req-c42ccc78-735d-43fb-8305-c706637c677c'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:49.720485 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] The token cache is being invalidate because project 761548f6db7e49e8a858ab97d77fbf90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:49.720829 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:49.721464 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:49.723197 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1444/2866] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:48 2022] DELETE /identity/v3/projects/761548f6db7e49e8a858ab97d77fbf90 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:49.727383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:49.737405 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686332-dd6d-4ec8-af11-6ae3adcd88a1 service neutron] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:49.773863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:49.802848 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:49.803489 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:49.803772 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:49.804049 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] PATH_INFO: `/v3/users/17fc9fa751164e2ca6062a7e3e49ed73` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:49.805458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fc9fa751164e2ca6062a7e3e49ed73)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:49.807625 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:50.228913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fc9fa751164e2ca6062a7e3e49ed73'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:50.229566 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fc9fa751164e2ca6062a7e3e49ed73'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:50.241148 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fc9fa751164e2ca6062a7e3e49ed73'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:50.427582 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fc9fa751164e2ca6062a7e3e49ed73'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:50.435413 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-41525bbe-7177-4b52-a657-05f19e2bf72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fc9fa751164e2ca6062a7e3e49ed73'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:50.441373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1423/2867] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:49 2022] DELETE /identity/v3/users/17fc9fa751164e2ca6062a7e3e49ed73 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:51.237976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:51.240151 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:51.253195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ccc78-735d-43fb-8305-c706637c677c admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:51.313722 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:51.314359 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:51.314669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:51.314953 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] PATH_INFO: `/v3/projects/3a533d25f80d46ffae960c242c30c530` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:51.316068 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a533d25f80d46ffae960c242c30c530)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:51.318648 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:51.576542 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a533d25f80d46ffae960c242c30c530', 'request_id': 'req-68649959-1ee9-4b45-86e9-e18f169af499'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:51.581801 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a533d25f80d46ffae960c242c30c530', 'request_id': 'req-68649959-1ee9-4b45-86e9-e18f169af499'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:51.810519 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a533d25f80d46ffae960c242c30c530', 'request_id': 'req-68649959-1ee9-4b45-86e9-e18f169af499'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:51.822793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] The token cache is being invalidate because project 3a533d25f80d46ffae960c242c30c530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81761) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:51.823090 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:51.823769 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-68649959-1ee9-4b45-86e9-e18f169af499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:51.825252 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1445/2868] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:51 2022] DELETE /identity/v3/projects/3a533d25f80d46ffae960c242c30c530 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:51.830165 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:51.830683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:51.830683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:51.850797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:51.852013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:51.914822 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1424/2869] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:51.922761 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:51.923058 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:51.923367 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:51.944195 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:51.945422 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:51.953657 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1446/2870] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:51.959015 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:51.969138 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158f083-9a0a-421c-aa37-c240b49adaa9 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:51.974499 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:51.975126 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:51.975427 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:51.975705 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:51.976396 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81762) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:31:51.976692 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:51.978335 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:51.991249 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1425/2871] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 06:31:51.995569 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:52.005749 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291fb78-6856-4527-892c-df18c1170de2 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:52.010576 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:52.011212 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.011495 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.011889 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.012523 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:52.014437 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:52.263585 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1447/2872] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:51 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:52.268067 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:52.279072 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e84bc61-9da7-4791-bfd8-68bc69e5c6ad admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:52.320193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:52.339309 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:52.339913 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.340242 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.340515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.341176 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:52.343121 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:52.616008 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1426/2873] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:52 2022] POST /identity/v3/users => generated 294 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:52.620099 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:52.630910 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c189727-8e21-4805-9603-afd08aca9e1d admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:52.635693 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:52.655706 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1448/2874] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:52.658929 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:52.670002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59f6ef-de51-499f-96cd-c1edaa575e22 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:52.681183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:52.681967 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.682683 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.683022 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.683783 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:52.685456 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3144b0-2faa-443c-a371-e7a486748395 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:52.697580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1427/2875] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:52.702082 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:52.713692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2f2da-4cf3-4b7b-8be3-d94e62e7d5d3 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:52.718444 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:52.719139 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.719424 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.719699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] PATH_INFO: `/v3/projects/9f53770e29844874b0e7c39a4b969278/users/449a2af046bf4ea9975cfd32ec43fa60/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.731797 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:52.738018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f53770e29844874b0e7c39a4b969278, user_id=449a2af046bf4ea9975cfd32ec43fa60, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:52.741921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:52.941755 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1449/2876] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:52 2022] PUT /identity/v3/projects/9f53770e29844874b0e7c39a4b969278/users/449a2af046bf4ea9975cfd32ec43fa60/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:52.952611 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:52.953116 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:52.953244 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:52.981044 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] MFA Rules not processed for user `449a2af046bf4ea9975cfd32ec43fa60`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:52.982269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:53.047315 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1428/2877] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:53.052819 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.117366 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022e3bb-dae4-4a78-8326-cfd6925f6735 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.121952 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.122559 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.122870 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.123145 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.123780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.125692 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:53.343737 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1450/2878] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:31:53 2022] POST /identity/v3/projects => generated 387 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:53.348580 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.360070 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76145dfa-df48-4260-ac4c-8ee8c302d863 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.420269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.420889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.421172 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.421449 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.422144 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.424094 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:53.536775 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1429/2879] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:31:53 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:31:53.540911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.551947 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68b05-9ce5-4ac2-a86d-5fd93c9d2b26 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.556865 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.557532 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.557818 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.558096 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.558779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.560381 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:53.572803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1451/2880] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 06:31:53.577003 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.587963 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b6848-0869-4228-a77f-4dd73d625f2f admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.592581 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.593237 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.593515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.593838 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] PATH_INFO: `/v3/projects/364274e9fa61425eb6e72e02e69748d4/users/94a7a33d3d1240339349c9deb5ee9082/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.609183 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] RBAC: Authorizing `identity:create_grant(project_id=364274e9fa61425eb6e72e02e69748d4, user_id=94a7a33d3d1240339349c9deb5ee9082, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.611889 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:53.874004 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1430/2881] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:53 2022] PUT /identity/v3/projects/364274e9fa61425eb6e72e02e69748d4/users/94a7a33d3d1240339349c9deb5ee9082/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:53.878094 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.889259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a253f-f9ca-49a8-83e6-a46a22c070b2 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.953843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.954461 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.954803 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.955093 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.955980 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.957639 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316aeb6-e4e8-47cb-944b-8a4f6aa848a8 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:53.968545 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1452/2882] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:31:53.973752 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:53.984589 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d121b3b-4003-4611-8564-ed2386c8441b admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:53.989393 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:53.990056 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:53.990342 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:53.990703 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] PATH_INFO: `/v3/projects/364274e9fa61425eb6e72e02e69748d4/users/94a7a33d3d1240339349c9deb5ee9082/roles/b4c6ab88b8d64fcdacfc6c5268120e8d` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:53.992662 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364274e9fa61425eb6e72e02e69748d4, user_id=94a7a33d3d1240339349c9deb5ee9082, role_id=b4c6ab88b8d64fcdacfc6c5268120e8d)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:53.995880 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:54.002706 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:54.008023 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1431/2883] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:31:53 2022] PUT /identity/v3/projects/364274e9fa61425eb6e72e02e69748d4/users/94a7a33d3d1240339349c9deb5ee9082/roles/b4c6ab88b8d64fcdacfc6c5268120e8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:31:54.019454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad238-efca-4437-91da-78a132e49365 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:54.020505 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad238-efca-4437-91da-78a132e49365 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:54.021018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ad238-efca-4437-91da-78a132e49365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:54.051311 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-e35ad238-efca-4437-91da-78a132e49365 None None] MFA Rules not processed for user `94a7a33d3d1240339349c9deb5ee9082`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:54.052547 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-e35ad238-efca-4437-91da-78a132e49365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:54.102205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:54.123730 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1453/2884] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:54.135398 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:54.137083 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:54.197193 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9ea25-a71a-492e-80c3-bd2ef60a6d96 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:54.204261 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:54.205379 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:54.205905 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:54.206383 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:54.217465 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:54.218935 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d4a76-5ee6-40e9-8faa-698ce79e066a service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:54.229145 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1432/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:56.265641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ad238-efca-4437-91da-78a132e49365 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:56.353740 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ad238-efca-4437-91da-78a132e49365 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:56.358389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:56.359072 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:56.359363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:56.359641 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] PATH_INFO: `/v3/users/449a2af046bf4ea9975cfd32ec43fa60` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:56.360776 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=449a2af046bf4ea9975cfd32ec43fa60)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:56.362778 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:56.663010 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449a2af046bf4ea9975cfd32ec43fa60'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:56.663798 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449a2af046bf4ea9975cfd32ec43fa60'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:56.677293 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449a2af046bf4ea9975cfd32ec43fa60'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:56.809659 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449a2af046bf4ea9975cfd32ec43fa60'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:56.818086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449a2af046bf4ea9975cfd32ec43fa60'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:56.824216 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1454/2886] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:56 2022] DELETE /identity/v3/users/449a2af046bf4ea9975cfd32ec43fa60 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:56.828638 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:56.829137 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:56.829251 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:56.850460 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:31:56.851699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:56.913431 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1433/2887] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:31:56.923018 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:56.924484 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:56.983669 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956deac-2b3a-49e6-a044-4747f8484c72 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:56.987363 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:56.987971 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:56.988284 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:56.989177 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:56.998417 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:56.999883 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:57.009632 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1455/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:31:57.600534 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:57.602047 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:57.616555 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa6a8e-c7d6-40e6-80b9-9b96fc9dc9d8 None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:57.621339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:57.621995 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:57.622271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:57.622567 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] PATH_INFO: `/v3/projects/9f53770e29844874b0e7c39a4b969278` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:57.623922 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f53770e29844874b0e7c39a4b969278)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:57.625944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:57.823304 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f53770e29844874b0e7c39a4b969278', 'request_id': 'req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:58.244626 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f53770e29844874b0e7c39a4b969278', 'request_id': 'req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:58.251759 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f53770e29844874b0e7c39a4b969278', 'request_id': 'req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:58.266231 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] The token cache is being invalidate because project 9f53770e29844874b0e7c39a4b969278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:31:58.266700 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:58.271150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:58.271150 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1434/2889] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:57 2022] DELETE /identity/v3/projects/9f53770e29844874b0e7c39a4b969278 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:58.274898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:58.284932 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441a65-6381-45d5-8987-473d9ab42db6 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:58.289185 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:58.350404 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:58.351140 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:58.351430 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:58.351731 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] PATH_INFO: `/v3/users/94a7a33d3d1240339349c9deb5ee9082` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:58.352898 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a7a33d3d1240339349c9deb5ee9082)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:58.354861 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:31:59.075119 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a7a33d3d1240339349c9deb5ee9082'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:59.075794 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a7a33d3d1240339349c9deb5ee9082'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:59.089376 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a7a33d3d1240339349c9deb5ee9082'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:59.249994 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a7a33d3d1240339349c9deb5ee9082'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:59.258957 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8fc2d190-cc48-4d0a-8681-55472d000876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a7a33d3d1240339349c9deb5ee9082'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:31:59.265391 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1456/2890] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:31:58 2022] DELETE /identity/v3/users/94a7a33d3d1240339349c9deb5ee9082 => generated 0 bytes in 991 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:31:59.806373 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:31:59.807993 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:31:59.828403 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851b388-71a9-4fb4-a6f7-a2f0ee86f4e7 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:31:59.858414 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:31:59.892200 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:31:59.892850 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:31:59.893133 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:31:59.893435 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] PATH_INFO: `/v3/projects/364274e9fa61425eb6e72e02e69748d4` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:31:59.895491 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=364274e9fa61425eb6e72e02e69748d4)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:31:59.897802 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:00.148128 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364274e9fa61425eb6e72e02e69748d4', 'request_id': 'req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:00.354878 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364274e9fa61425eb6e72e02e69748d4', 'request_id': 'req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:00.360908 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364274e9fa61425eb6e72e02e69748d4', 'request_id': 'req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:00.372697 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] The token cache is being invalidate because project 364274e9fa61425eb6e72e02e69748d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:32:00.372984 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:00.373699 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-e6e4a228-20d3-4b1e-9ba0-4d1148525fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:00.375241 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1435/2891] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:31:59 2022] DELETE /identity/v3/projects/364274e9fa61425eb6e72e02e69748d4 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:32:00.380265 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.380911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.380911 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.401929 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:32:00.403129 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:00.464192 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1457/2892] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:32:00.472269 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.472564 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.473458 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.493658 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:32:00.494944 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:00.503220 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1436/2893] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:32:00.507793 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:00.517805 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b743e3d-6559-4885-aea0-7c4cdac81ca7 None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:00.523259 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:00.523881 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.524167 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.524446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] PATH_INFO: `/v3/domains` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.525134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81761) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 06:32:00.525446 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:00.527124 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:00.540753 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1458/2894] 10.222.0.40 () {58 vars in 1153 bytes} [Fri Aug 26 06:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 06:32:00.544953 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:00.555964 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70fe27-b4ad-4f3b-8337-df24148db48a None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:00.560786 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:00.561420 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.561702 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.561976 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.562606 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:00.564553 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:00.748858 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1437/2895] 10.222.0.40 () {60 vars in 1153 bytes} [Fri Aug 26 06:32:00 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:32:00.753642 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:00.764409 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b404-5a58-46cc-94a7-56a744ceb72f admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:00.773886 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:32:00.827122 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:00.827765 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.828028 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.828307 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] PATH_INFO: `/v3/users` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.828965 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:00.830959 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:00.959915 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1459/2896] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Aug 26 06:32:00 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 06:32:00.964295 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:00.976267 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec74bc-0fc9-466e-8021-6aeee6acbd42 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:00.980971 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:00.981578 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:00.981863 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:00.982132 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:00.982809 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:00.984416 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:00.995824 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1438/2897] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:32:01.000101 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:01.010667 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8a70-375b-45c5-a844-3bcb32e4d188 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:01.016174 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:01.016786 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:01.017107 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:01.017389 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:01.018040 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:01.020428 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee94d5-14af-406d-9f67-9c01aa85f129 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:01.031699 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1460/2898] 10.222.0.40 () {58 vars in 1122 bytes} [Fri Aug 26 06:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 06:32:01.036729 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:01.048234 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06131a-3f31-4c95-b359-2a28ddf56195 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:01.053339 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:01.053875 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:01.054648 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:01.055202 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] PATH_INFO: `/v3/projects/fa4bab667b10408e862103b5e23298e7/users/662c70b92b0248da96ad90bd751a61cb/roles/d14184d6cd4d4614b284d4c9394bf9ab` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:01.071387 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4bab667b10408e862103b5e23298e7, user_id=662c70b92b0248da96ad90bd751a61cb, role_id=d14184d6cd4d4614b284d4c9394bf9ab)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:01.075701 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:01.245218 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1439/2899] 10.222.0.40 () {60 vars in 1483 bytes} [Fri Aug 26 06:32:01 2022] PUT /identity/v3/projects/fa4bab667b10408e862103b5e23298e7/users/662c70b92b0248da96ad90bd751a61cb/roles/d14184d6cd4d4614b284d4c9394bf9ab => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:32:01.258193 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] REQUEST_METHOD: `POST` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:01.258454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:01.258682 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:01.288849 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.auth.core [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] MFA Rules not processed for user `662c70b92b0248da96ad90bd751a61cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81761) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:32:01.290331 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:01.361445 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1461/2900] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:32:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:32:01.372954 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:01.374725 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:01.436515 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fcd12-b1ed-4011-91a6-a8f1042fe603 admin admin] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:01.441090 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4710a6cf72564ff7abf8d35fa8e90a5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:01.442368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] REQUEST_METHOD: `GET` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:01.442917 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:01.443196 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:01.451089 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:32:01.454063 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:01.456845 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dc69e8-69d8-49e4-a3b4-9056082bf22b service cinder] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:01.469490 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1440/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:32:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:32:11.358608 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:32:12.335898 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=2 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:32:15.404134 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:15.469528 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b445d2-0faf-4155-ab32-e5737a50dd4e None None] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:15.474976 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:15.475634 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:15.475921 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:15.476205 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] PATH_INFO: `/v3/users/662c70b92b0248da96ad90bd751a61cb` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:15.477411 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662c70b92b0248da96ad90bd751a61cb)` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:15.479467 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:15.699652 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662c70b92b0248da96ad90bd751a61cb'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:15.700328 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662c70b92b0248da96ad90bd751a61cb'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:15.713308 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662c70b92b0248da96ad90bd751a61cb'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:15.849858 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662c70b92b0248da96ad90bd751a61cb'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:15.857809 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.notifications [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662c70b92b0248da96ad90bd751a61cb'} {{(pid=81761) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:15.864589 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1462/2902] 10.222.0.40 () {58 vars in 1224 bytes} [Fri Aug 26 06:32:15 2022] DELETE /identity/v3/users/662c70b92b0248da96ad90bd751a61cb => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 06:32:15.869318 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] REQUEST_METHOD: `POST` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:15.869814 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:15.869950 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:15.888951 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.auth.core [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] MFA Rules not processed for user `d4b059129fcd4293b7ae6f76c1ca5b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81762) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 06:32:15.890280 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:15.957368 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1441/2903] 10.222.0.40 () {58 vars in 958 bytes} [Fri Aug 26 06:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 06:32:15.967729 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:15.969336 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:16.033600 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d518bce-56a1-434f-8f9b-5f7dd5224427 admin admin] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:16.038429 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddd1cf9d4721422194e5ec876b819033', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:16.039086 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:16.039372 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:16.039651 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:16.049923 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:16.051454 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:16.062999 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1463/2904] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:32:16.634479 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] Authenticating user token {{(pid=81762) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:32:16.636002 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.fernet_utils [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81762) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:32:16.649591 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa25e-902f-42fe-a342-101513c1ebbb None None] Validating token access rules against request {{(pid=81762) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:32:16.654857 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4b059129fcd4293b7ae6f76c1ca5b3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6316e285f13041a3ae4690b463c7dfa7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81762) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:32:16.655541 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:32:16.655826 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:32:16.656271 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] PATH_INFO: `/v3/projects/fa4bab667b10408e862103b5e23298e7` {{(pid=81762) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:32:16.658184 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4bab667b10408e862103b5e23298e7)` {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:32:16.661013 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] RBAC: Authorization granted {{(pid=81762) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:32:16.822797 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4bab667b10408e862103b5e23298e7', 'request_id': 'req-95bda165-648b-41be-a8ca-cb5569e59cc0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:16.952721 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4bab667b10408e862103b5e23298e7', 'request_id': 'req-95bda165-648b-41be-a8ca-cb5569e59cc0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:16.959708 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4bab667b10408e862103b5e23298e7', 'request_id': 'req-95bda165-648b-41be-a8ca-cb5569e59cc0'} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:16.970939 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] The token cache is being invalidate because project fa4bab667b10408e862103b5e23298e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81762) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 06:32:16.971222 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:16.971860 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG keystone.notifications [None req-95bda165-648b-41be-a8ca-cb5569e59cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81762) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 06:32:16.974181 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: [pid: 81762|app: 0|req: 1442/2905] 10.222.0.40 () {58 vars in 1233 bytes} [Fri Aug 26 06:32:16 2022] DELETE /identity/v3/projects/fa4bab667b10408e862103b5e23298e7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 06:32:26.058054 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:32:26.967859 dv-cinder-853910-3-iscsi devstack@keystone.service[81762]: DEBUG dbcounter [-] [81762] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81762) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 06:34:45.429209 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] Authenticating user token {{(pid=81761) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 06:34:45.430950 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:34:45.575357 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c55f5-1e30-4a20-975e-aae9d3b62576 service neutron] Validating token access rules against request {{(pid=81761) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 06:34:45.633842 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48e76f49c2224e959267952f79669906', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdab9db7f04e4686b082a3ca17fb9d07', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81761) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 06:34:45.634452 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] REQUEST_METHOD: `GET` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 06:34:45.634780 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 06:34:45.635064 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81761) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 06:34:45.636831 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.fernet_utils [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81761) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 06:34:45.648779 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 06:34:45.650387 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1026c-0792-4862-8621-91b3941bc8e6 service placement] RBAC: Authorization granted {{(pid=81761) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 06:34:45.661405 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: [pid: 81761|app: 0|req: 1464/2906] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Aug 26 06:34:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 06:34:55.656843 dv-cinder-853910-3-iscsi devstack@keystone.service[81761]: DEBUG dbcounter [-] [81761] Writing DB stats keystone:SELECT=89 {{(pid=81761) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}